JOIN
Get Time
forums   
Search | Watch Thread  |  My Post History  |  My Watches  |  User Settings
View: Flat (newest first)  | Threaded  | Tree
Previous Thread  |  Next Thread
Hiding the submitter details in contest page until the end of appeals response phase. | Reply
In the old theme, The submitter details will not be shown in the contest page. But in the new theme, this is visible from which the reviewers can see who has submitted and when they submitted for a challenge.

This becomes really problematic when there is a highly established submitter among the other new submitters.

I think this gives unfair advantage to the highly rated submitter over the new submitter. Is it possible to hide these details like old theme?

And in the old Topcoder we had rules like we should not include name or handle in the submission. Does the rules still apply? I think we need those rules for fair competition.
Re: Hiding the submitter details in contest page until the end of appeals response phase. (response to post by leo.ananth) | Reply
Thanks leo.ananth for bringing this up. We are discussing this internally. I will give my opinion here so people can agree or disagree with it.

Personally I think if the review board is unfairly favoring people that is a different issue and we should address that by monitoring for it and punish/remove offending members.

I think the idea of anonymity is tough. I personally look at a lot of submissions and can usually guess regular competitors submissions. I would like for us to start requiring videos (OK not to have sound) for some challenges (not all). Seeing similar desktops or accidently including your name in a template can give away anonymity and I don't think should be punished.

Now that said maybe there is a happy medium where we dont make it easy but we don't punish people for accidently including personal info.
Re: Hiding the submitter details in contest page until the end of appeals response phase. (response to post by mess) | Reply
mess,

I think proving that a reviewer is favouring a submitter would be difficult.

The problem I see now is shallow review. I understand that rating is given for the reviewer by copilot based on his review performance. But that will not affect the reviewer if the track has very few reviewers.

I can point out the reviewer who is consistently doing shallow review which is evident from his review rating history but continues to get review opportunities due the the low number of reviewers available. The reason why I am requesting the submitter details to be hidden is, reviewers who are doing shallow review is always making sure that the rated members are scored higher than the remaining submitters assuming that they are going to win anyway and to match their score with the other reviewers. And when there are only two submissions, this is the case always.

Shall I Email you the specific details?
Re: Thread "Hiding the submitter details in contest page until the end of appeals response phase." in category "Round Tables" has been updated by leo.ananth (response to post by leo.ananth) | Reply
Please feel free to send specific details into support@topcoder.com and
we'll get people looking into specifics.

In general, I agree we need to find a better way to reward more thorough
reviews. The scorecard itself probably needs some specific tweaks in order
to encourage more in depth reviewers. I'd love to hear other ideas as well.
Re: Thread (response to post by mess) | Reply
i would like to point out one more thing i feel unhappy/irritated about usually after seeing my review scorecards although it is not strictly related to this thread but is close enough ...

i think we need to define just how important "matching the storyboard" is in UI prototype . if you pick up 'n' review scorecards, you would find this phrase("does not match the storyboard") at least 3*n times ....and most often they are just a few pixels here and there.
i agree that layout should match the storyboard, but there should be some margin of error allowed , otherwise at least i constantly find myself worried too much about pixel matching rather than logic or quality of code ...

and sometimes the storyboard themselves are not proper/good quality and we can't keep confirming each and every thing in the forums ...

and yes, about the particular reviewer leo.ananth is talking about, i think everyone actively participating in the UI prototype track is aware of who that reviewer is ...but he usually gives a pretty good score ...so
Re: Thread (response to post by daga_sumit) | Reply
We have already addressed the review item. What do you suggest for the scorecard? It would be great for you to give examples that can be discussed.
Re: Thread (response to post by mcards13) | Reply
For starters i think the co-pilot should let the reviewers know that some margin of error should be allowed as far as spacing (margin/padding) is concerned ...unless the co-pilot is really keen on matching the storyboard pixel by pixel

and another thing co-pilot could do is (i am assuming that most of the times the co-pilot who launches the UI prototype challenge must also have been the co-pilot for the design phase), point out in-consistencies(as many as he/she can) in the storyboard after the challenge is launched. It will be easy for him/her because he/she would already be familiar with the storyboard (above assumption) .....we could clarify these things in the forum ...but i feel this approach will make it easier for everyone....

although these seem like small things and the current system is fair for everyone but i would rather not worry too much about pixel matching and who does not love consistency :)
Re: Hiding the submitter details in contest page until the end of appeals response phase. (response to post by leo.ananth) | Reply
No offense, but just my personal inner feelings.

Reviewer has unfair review:
-- Agree with leo.anath it is very difficult, But I find "Escalation" process and I have seen it working. Not only because reviewer was favoring but was missed some bad points in winner and saw other thing in second one, when the second contestant re-appealed with his points(including winners submission and his) he was declared as winner lately.
So what I believe is if reviewer is doing review pixel by pixel, we submitter has the option to review pixel by pixel both submission and escalate the against the reviewer if it is unfair review. Definitely his review has to be done pixel by pixel.

hiding the submission:
Personally I do write my handle in the code itself as author, I don't think hiding your submission will really help. If you think some reviewer is favoring to A, then we can't stop (submitter can send the submission id) unless we report to the copilot/PM/admin and upon verification reviewer must be removed from the review board.

So, do your best with your identity, be honest and report to concerned one.
RSS