Exercism v3 launches on Sept 1st 2021. Learn more! ๐Ÿš€๐Ÿš€๐Ÿš€
News, interviews and articles from the Exercism community.

Dealing with oversubscribed tracks

One of the biggest challenges we face is tracks getting too popular and our mentors being unable to cope with the volume of submissions. The wait time for students increases, and mentors have the very unsatisfying experience of seeing queues keep growing despite putting a huge amount of work in to try and keep on top.

Solving this is far from trivial, but we believe we have some good solutions that will make things better (see our recent blog post for details). However, these will all take time to implement, and there is a very real problem today.

We've therefore decided to add an "Oversubscribed" status to tracks, where when the median wait time for core exercises becomes too long, we:

  • Only allow students joining a track to use Practice Mode, not Mentored Mode
  • Stop students in Practice Mode from requesting mentoring on their solutions (they can continue to use Practice Mode independently)

Once a track gets its median wait time back under control, the system will automatically allow new students to join Mentored Mode, and allow Practice Mode students to submit solutions for mentoring.

These changes have no effect on students already working in Mentored Mode.

There are lots of extra bits of functionality we could add to this (e.g. queues to join, notifications when tracks unlock), but for now we're just taking a somewhat brute-force approach to fix the main pain point.

Hopefully this will give a better experience and more realistic expectations to students, remove the pain of ever-growing queues for mentors, and allow us to focus on implementing the bigger changes that will make a more long-term impact.

Avatar of author
Jeremy Walker
Co-founder of Exercism. I'm an entrepreneur and software developer, and have been running a variety of businesses and non-for-profits for the last decade in the fields of medicine, education and artificial intelligence.
View Profile

Community Comments

Great idea. Just stumbled upon this when I subscribed to the Nim track. Oh well :)

Ha, I just stumbled upon this when I subscribed to the Nim track.

3 minutes after you, Oleksii. :-D

I would recommend marking tracks as "Oversubscribed" on the Language Tracks page to help us with initial selection. This may also help to alleviate the problem.

Really good idea. You have an amazing community here, having just joined this site and having done a lot of online learning, I've been super impressed with the quality of feedback. The last thing you want is to dilute that.

From a users perspective, I think waiting for a slot to open up for mentoring is fine, as it'd increase the importance of being 'connected' with a mentor. It'd sort of gamify 'earning' a mentored spot. However, adoption of the site might slow down?

Some ideas I have

  • Timeout on current mentored students, so that if they don't engage then they're put in the queue to free it up for a more active user.
  • Allow 1 mentored solution to start, for people to see the value in mentoring and then let them know there is more where that came from, theres just a queue.

Great idea! And I just got here because it had that on Swift :) Something related, maybe: sometimes a track is basically dead. For example, on Racket, it says "The median waiting time for mentoring on this exercise is about 1 month". Well, that sounds like I'm going to switch to practice mode... same thing with CoffeeScript.

This message has been deleted

Thank you for the very clear explanation! Love this platform :)

Where can we join the discussion on how to improve the queue times?

One idea I have is to allow students that have already had their solution approved by a mentor to mentor solutions to the same exercise, if the queue time is past a certain threshold.

From my experience many students make the same errors on the same exercises, so you spend a lot of time giving the same advice over and over.

By including students as multipliers you can reduce the amount of repetitive work for mentors, reduce queue times and enable students to learn to give constructive peer reviews.

In addition some of the students might become interested in becoming normal track mentors, if they have built up some confidence through peer reviews.

Yes! See one, do one, teach one.

It might be as well to provide a warning when switching to practice mode: this change might not be easily reversible! I switched to practice mode on the rust track to have a look at the list of exercises - now I'm stuck there with no idea when I'll be able to go back to mentored mode.