What “Testsolving” Means

Over the past few years I’ve been asked to testsolve contests several times, and for some reason or another it inevitably becomes “please help me write my contest”, or worse, “please write the rest of my contest for me”. I suspect there are two reasons for this:

• people don’t understand what “testsolving” means,
• or they are too irresponsible to write their contest themselves.

If I suspect you are the second type of person, I will just flat-out refuse to help you with your contest. But I suspect there are a lot of hard-working people who may unintentionally be taking advantage of their “testsolvers” because of misunderstandings.

How do you write a contest?

• Write problems. At some point, possibly after arranging them into a test but no later, you must write full solutions for each of them.
• Arrange problems into a test. The people who do this can be the people who write problems, but this is not always the case.
• Verify feel. Are the problems appropriately easy/hard for the intended target audience? Is the “intended” target audience who you will actually get? Are the subjects distributed reasonably? Are the problems fun to do? Iterate until the answer to all these question is yes.
• Verify correctness. Make sure every problem is well-defined, answers are what you think they are, and the solutions are clear and logically sound.

Particularly for verifying correctness, it is imperative that you have full solutions written. Verifying correctness does not mean finding the answers to your problems. It does not mean verifying your answer key, either. It means you verify that absolutely everything is correct.

You aren’t ready to ship a contest until you’re ready to release everything right now, meaning the test itself and the solution manual. It maybe even means preparing some advertising materials, or the means of testing (testing venues for in-person, testing portal for online).

So when do testsolvers come into play? At earliest it is at the third step, verifying feel. Personally I make a rule for myself not to involve testsolvers until I think I’m ready to ship a contest. (More precisely, by “I” I mean Math Advance.) Testsolvers are just there to make sure that the contest is ready. They are the final correctness verification.

One-line summary: If you expect your contest to change after receiving feedback from someone, they are not a testsolver, they are helping you write your contest.

But I need help on my contest!

Fair enough. Testsolving is not the only role non problem writers can take, and it’s perfectly normal to ask for help, even from outsiders to your group. Just don’t tell people you want them to “testsolve” if you’re giving them an obviously flawed product to look at. Testsolving means that you have made your very best efforts to create a final version of your contest, and that you are very sorry if anything has to change at all.

Obviously you won’t be able to catch everything, and you may need slight assistance or want feedback on earlier steps, such as “what about this new arrangement of problems?” or “is the feel better now?” It is okay to ask for these things. But remember, the job of a testsolver is not to help you write your contest. They are just the final verifiers. Inevitably your mistakes will be revealed, but the thing about mistakes is you’re supposed to put some effort into avoiding them.

Don’t ask me to testsolve until…

…you have all your solutions written. No exceptions. I don’t want to read a test half-filled with problems that make zero sense. I mean, you should be careful enough that you don’t even put problems on your test unless you know they are actually comprehensible. Everyone in Math Advance can do it, it’s really not that hard.

But if you know that you’ve been cursed with the inability to properly express your mathematical ideas, then one way to make sure it doesn’t affect me is by writing solutions. It’ll be really hard to write clear solutions if the problem isn’t clearly stated, so there’s that.

If you often get the feedback “I don’t understand this problem”, one thing I haven’t tried but recommend is having at least one solution per problem by someone who isn’t the problem author. That way if a problem is unclear, it becomes apparent quickly.

I’m not ready for testsolving, can you still help?

Do not ask me for help unless you are using mapm. If my time is not valuable enough for you to use a totally new tool for creating your contests, that is totally fair. But my time is worth that much to me, and I am sick of reading disorganized, carelessly written Overleaf documents that are not even in sync with each other.

For those of you who are willing to use mapm :) If you want me to help you write your test, the answer is probably still no. I just don’t write enough problems to do that, period. If you want help with arranging your test, maybe, with the caveat that I will not take a leading role. I am pretty opinionated and vocal though, so you will definitely get lots of feedback.

If you are willing to use mapm but don’t know how or want me to help you set it up, I’d love to do so.

If you have any questions about contest-writing, feel free to email me. If it’s a good question, I might just update this article to include it!

Update (2022-07-27): Please ask your testsolvers to look through your contest before you publicly announce it! If your contest is significantly flawed, in 99% of cases the “testsolver” will end up taking more responsibilities (because anyone who volunteers their free time to testsolve a contest is probably really nice). You don’t want to force a time crunch on them that they never signed up for.

The alternative is that the testsolver goes “fuck it, I don’t care anymore” and BSes through the rest of the test or just stops commenting. This happens a lot with contest creators who are either unnecessarily stubborn or fail to plan ahead, and you don’t want this to happen to you either.