||This discussion started in another thread.
My feeling is that marathon matches improved in the last few years, both in terms of number of matches (sponsored and "fun") and quality (better problem descriptions, server stability), but the number of active competitors declined.
I think the website changes, widely discussed in other threads, are responsible for most part of this reduction of interest in Marathon Matches.
I collected some data, from the last 7 years, to check if the participation level really declined:
| Fun Avg. | Sponsored Avg. | TCO Qual. Avg.
| Matches Competitors | Matches Competitors | Rounds Competitors
2011 | 2 201 | 5 103 | 3 275
2012 | 1 163 | 20 79 | 3 152
2013 | 2 144 | 11 84 | 3 296
2014 | - - | 19 60 | 3 225
2015 | 5 77 | 12 51 | 3 175
2016 | 2 47 | 16 53 | 3 140
2017 | 6 59 | 12 38 | 4 116
Notes: TCO Championship rounds were excluded;
TCO Qualification rounds were merged to "mirror" matches;
Couple of test / private / educational matches were ignored.
I would like to hear practical suggestions of how to increase the number of competitors back again, and to improve the track in general.
EDIT: Updated on January 6, 2018, with 2017 final numbers.
||Some brainstorming ideas.
Make it easier for 1st time competitors:
- Provide a basic solution with the visualizer.
- Put a reminder post at the beginning of the MM forum on the submission format (the best information I know is a forum post by Nikolas), plus few links to useful information about MM.
- Create an everlasting easy beginner MM, with template solutions in major languages, just to ease the understanding of the submission/scoring process.
Do some announcements on Codeforces, Codechef, ...
A YouTube teaser of the visualizer testing one solution.
For sponsored MM:
- Some general press releases, and announcement in forums specialized on the subject.
- Some algorithmic/optimization MM with smaller computation time.
||Probably I am one of those few who will disagree that the new design is worse than the previous one. I recall times when I first opened the topcoder website in 2005 or so and it looked really cumbersome to me. It was hard to find how to compete in algorithm competitions and where to compete. I do not say it became easier with new design.
So, what happened is, as can be seen from the table, the (fun) Marathon Matches were mostly discontinued during 2011-2014. The old marathoners probably found some other places to play, or retired, while a little (if any) new members appeared here (because of low amount of fun matches per year).
Thus, there are basically 2 possibilities on how to increase participation:
- try to bring back old members (hard to do);
- try to find new members that participate on a +/- regular basis (easier).
My opinion is that the 2nd way could be done for example when there is a TCO Qualification. Often new members just give up after their first competition, so maybe it would be beneficial to continue these "Learning data science" series and even recommend to participate in them before TCO.
||klo86min, great ideas!
Providing a basic solution was on my list too :-)
Two other suggestions:
Early announcement: adding matches to the calendar, even with a tentative schedule, and sending an e-mail about upcoming matches.
Learning about the matches one week (or at least few days) before its start would help a lot.
Post-match Feedback: for sponsored matches, Topcoder could add another task to prize winners: making a small forum post, with a general idea of the solution.
Winners already have to write a solution description, with detailed information.
It would be a simpler and smaller version of that.
Without adding much trouble to the winners, it would provide some ideas to other competitors.
I think the post your approach thread is the one of important/fun things in regular matches that we usually don't see in sponsored matches.
||TCO Qualification is a big entry door.
However, it depends on good and early advertisement about TCO itself.
One thing that I noticed is that, since reaching the top 4 spots (advancers to onsite final) seems too hard, many people give up before trying.
Maybe introducing additional goals could help.
For example, for each MM qualification round, people would collect "points" based on their final rankings, to be used as odds in a random draw of prizes (a TCO trip?).
In a way that improving your score would also improve your chances.
||I must say the quality of the problems has been really great lately and the discussion that follows is very useful. So the problem is not here...
I don't think there is an easy way to increase participation level. In fact, we are fighting a losing battle. Like gorbunov said most marathoners have left to other sites or retired due to lack of fun matches. In my case, TopCoder used to be one of few places where I did marathon matches, now there are many that are equally good (or better): Al Zimmerman, Kaggle, Codingame. So now I only pick the ones that I am really interested in.
To make things better, an early announcement system would be really good. Often I miss the start of a MM and this is a significant disadvantage. If you miss the first few days then at my level there is no point competing in the match. Doesn't apply to your level of course :)
Easy way to start for beginners would help a lot too. I was avoiding MM matches for many years, because I thought they were too hard to start. A template in each language that just returns a valid low-scoring answer would be really good. Many other sites are doing this too.
Oh and please stop calling them "Data Science"!!!
||I tried few competitions on Codingame and I think the three reasons of the relatively high number of competitors on this website (almost 4000 for the last MM-like contest) are the following :
1) Early announcement : the announcement for the next competition take a big part of the homepage of the site, and this very early (several months before the competition).
3) Easier to start on a problem : just after registering on a contest, you just need few clicks to see a solution running on a visualizer. I think this is the big one. Competing (especially for the first time) on a MM on Topcoder requires some time commitment.
I don't know how easy is it to apply those ideas for Topcoder (since it has a different business model and technical platform) but that are three things that, I think, make a big difference.
||Great points eldidou. I would like to add one more. The multiplayer competitions are really fun and you see your bot competing against other bots almost in real-time. This makes the competition so much more exciting.
One more important point for attracting competitors. Codingame offers jobs and interviews with multiple international customers.
||Just some of my opinions (only about "fun" MMs - sponsored MMs can have their own custom rules, given that there are money prizes involved):
1) make the "fun" MMs shorter (e.g. 1 week): now it feels that one needs to spend most of the full 2 weeks of a MM in order to obtain a decent result - this is a pretty big time commitment for many people
2) organize more "fun" MMs (this can be done more easily if they are shorter) - if the issue is that it's not easy to come up with problems for more "fun" MMs, then I'm sure that the more experienced MM competitors can help with that (at least initially)
3) mandatory "post your approach" thread for every "fun" MM: if one didn't obtain a good result, then at the very least the MM will become a learning experience, which will help the person improve for future matches; but if there's no analysis from the top ranked competitors, then there's nothing to learn (reading the code source is hard), so the whole experience was not just frustrating, but also useless.
||I think that a lot of good points have been raised.
* First, I think that TopCoder could try to partner with some companies who have Discrete Optimization challenges (this is how I would classify fun MM's in an abstract sense) and have them sponsor events (just like Kaggle does for machine learning). What is unique about the fun MM's in my opinion is that their almost the only venue for serious NP-Hard problems. And there are companies out there with these kinds of challenges, so I can see some of them being interested to sponsor events. One example would be Uber. Obviously, keep the problem fun, but I'm sure they would be interested in recruiting some of the top performers.
* I agree that real-time multiplayer battles are exciting, the problem with them in my opinion is that most of the time they reduce to some variant of alpha-beta search. An example of another site with bot challenges is HackerEarth, and all 3 bot challenges hosted there were solved by alpha-beta :)
I didn't take part yet in Codingame, so I may be mistaken.
* I don't wholly agree about Kaggle and Al Zimmermann having better contests in the fun MM's space. Kaggle is purely machine learning, besides the end-of-the-year challenge. Al Zimmermann is twice (?) a year, with a completely different style of problems.
I think the main strength of the fun MM's is the huge variety in terms of techniques and approaches.
How to keep the variety while making it easier for first timers is the hard part :)
||Thank you for creating this thread wleite.
The popular opinion here seems to be about getting a notification for upcoming matches. I have added that in the excel and will discuss it in our upcoming CAB meeting with topcoder.
Re other points, I will wait for further discussion on them before bringing it to topcoder's attention.
||From a beginner's perspective,
1. Mandatory Post match discussions on the techniques used by the winners.
2. Some of the suggested areas for reading related to that marathon match when it starts(or during pre-registration)
3. A simple algorithm(similar to kaggle) to get started
||I'm not a TopCoder admin but still a huge fan of MM track, and I'm always interested in ideas that can increase its popularity and participation.
Some ideas in this thread look like they can be implemented with minimal admins' involvement: a writer can easily provide links to some reading material in the sticky post for the match, and (almost certainly) a set of trivial solutions in all languages. Codeforces announcement is also doable.
Putting matches on calendar early and sending email notifications are trickier. As a CAB member I've been fighting tooth and claw all past year to revive per-match SRM and Marathon emails, and while I've succeeded in SRM area, Marathons still get only a mention in data science newsletter. We'll see how this year's CAB does :-)
Finally, I'm really curious how people who suggest making approach descriptions by winners mandatory intend to enforce this? In matches with prizes describing winning approach can be made a prerequisite of getting the prize. But fun Marathons offer just that, fun.
||Participation will always go down, since people have to move on with their life, and contests won't always fit the schedule. So you have to attract new members.
Thus, ideas can be split into two areas:
1) Attracting new members.
2) Retaining new members.
I won't repeat what others have said. Otherwise, I'd have to make this a three page long rant about what I believe makes sense and what doesn't. And, this would be worthless since I don't know the crucial inner stats that TC has and the expected costs of implementing the features.
So instead, I'll add something new. IMNSHO, the single biggest issue is the lack of advertisement for TC. Even after acquisition by Appirio, nothing has really changed. TC almost exclusively relies on it's community for any kind of promotion. No press releases, no articles, no stabs at viral marketing, no promotion of its members, no connections with other competitive programming sites, no posts to social news sites (not most ethical, but lot's of people do it). Sure this takes time and resources, but without it's impossible to do anything, since most of the blog posts / videos on TC website have zero reach. And since most of us lost hope that TC will improve anything, relying on community is as good as doing nothing.
On the other hand, TC doesn't really care that much. It's clearly in the state where it tries to cut down it's losses. And well, SRMs, for-fun MMs, TCO, reaching more members are a kind of luxury goods when you look at it from financial perspective. TC lost most of it's active community by not caring about it. There are also no plans for using TC as a recruitment platform (someone asked about it during the TTL with Mike Epner), so it's very unlikely that TC would like to focus on rebuilding the community.
Google trends. This is a bit misleading for several reasons. Clearly data from India/Bangladesh completely dominates the outcome, but I'd blindly guess that those are very important regions in terms of spreading the word. What's depressing is that all of the sites are getting more attention while TC goes down.
Reddit mentions: number of google hits for "site:reddit.com XXX". topcoder (2030), codeforces (2370), hackerrank (5180), codechef (2990), kaggle (17100). Not ideal, and much more US-heavy (thus the domination of kaggle), but it gives another viewpoint. I'd imagine that most of TC mentions are actually quite old.
||I would like to point out something from the google trends that support the Idea of Psyho.
You can notice that Egypt has one of the higher hit rates for topcoder.
If you open the trends for Egypt (here) you will find the first peak at March 2013.
At that time TopCoder made a competition by nominations to attend TCO13 based on nominations from copilots.
The nominations were made by facebook likes. And there were two Egyptians in this competition.
Although, the "facebook likes" thing didn't go very well. But apparentely, this thing changed the whole trend for the in Egypt towards topcoder.
Again, it is the same idea. TC need to try innovative ideas to reach out to people.