The psychology of peer reviews!-A Data Scientist perspectives..What may have been wrongly interpreted

 The psychology of peer reviews! The psychology of peer reviews - S Anand (s-anand.net)

Quite an intriguing title for a post! This becomes even more special when a data scientist writes on human psychology! Anand S has categorized the personalities of the evaluating students discretely! This shall amuse and inspire you to think deeper than one "competitive" peer level event for the "same" task in a given setting which has "academic" value for a transient period on engagement!
Everyone in the set of ~500 students, who produced their analytics on 3 criteria: Insight, Visual Clarity, and Accuracy (with clear details on how to evaluate.) and everyone's work was evaluated by 3 different students from the same set, and they gave "individual evaluator's" ratings? from zero to 100% on each of the 3 criteria?
Based on how every student evaluated others (~145 visualizations work/student) , they were categorized as
1.Lazy
2. Lazy but smart
3. Extremists
4. Mild Extremists
5. Angry, and
6. Deviants
Just Lazy are the majority with 15% falling there! Deviants (3%) as defined are random evaluators so they can be just those who did no evaluations at all or just gave random scores, inconsequentially.
Angry (3%) and Lazy but smart (4%) also look like the minority in their views!

Some concerns that I have are here below-
1. Criteria called "Insight" seems too simplistic. They may have presented "Highlights" not Insights! Given that they got the case study and then data sets around the case to analyze and present its visualization, depending on what assumptions they made and what clear line of analysis they were asked to probe and present! I wonder if "Insights" can be presented as "data"!
Insights to me is the domain of business experts to deduct out of any data, how much sliced diced, tortured, etc, remain data! Insights are knowledge based and not any model-based outcome! Does DS promise "Insights"?
2. Though you may call all categories psychological types, but none of these words "Lazy, Lazy and Smart, Angry, Deviant, Extremists" are "psychological terms used to define personalities!
3. Are technical evaluations of such data visualization tasks, a space for "opinions" and "biases" to take over, it's pure material value? If Data Scientists were Personality" laced humans that can overshadow the sanctity and merits of a technical task, then we have a very big risk to avert!

Comments

Popular Posts