The Outstanding Gamification Awards Finalists
by Michael Wu
After approximately 3 weeks of scoring and hard work, my esteem panel of judges have finally chosen the finalists for the Outstanding Gamification Awards (OGA). There are some changes to the categories, selection process, and game mechanics due to the number of entries that we received and timing. And I like to be as transparent as I can, and keep the rest of the community in the loop of what we did and why we did it.
First, at the 10/18 deadline, we had a total of 21 OGA submissions. However, 2 of the categories (OGA for Inclusion and Diversity and OGA for Employee Engagement) only received a single submission. Consequently, we decided to keep the submission platform open for 1 more week. Not only did we receive a few more entries in the Inclusion and Diversity Category, we also got a few more late-submission for the other categories.
There were several duplicates entries that were disqualified. We also had to initially disqualify 1 entry that had a broken video link and 1 more that were over the time limit of 3 minutes. However, the Employee Engagement category still only had a single submission. So we decided to combine the Employee Engagement category with the Audience Engagement category, since Employees are simply a more specific type of audience. So we ended up with 25 submissions. across 6 categories, and the distributions were as follow:
As you can see that several categories have only 3 submissions. This means that if we choose the top 3 in each category as finalists, then every entry in these categories will automatically be selected. This defeats the purpose of having an initial round. Thus, we had to modify the finalist selection criterion. We will only choose the top 2/3 (i.e. the top 66.66%) or top 3 (whichever smaller after rounding) in each category to be the finalist. Basically:
- For the categories that have more than 3 submissions, the top 3 will be the finalists
- For categories that have only 3 entries, we will only choose the top 2 as the finalists
- For categories that have less than 3 submissions, we will have to either combine it with another category or withdraw the category due to low participation
The Number Crunching
We have 7 judges (including me) working across 3 continents, on top of their full-time jobs and family obligations to score all the entries. There is a lot of time and sweat that went into scoring all the entries. The judges were so generous that they even spend the time to score the entry that was disqualified by me because the video was longer than 3 minutes.
We are also extremely fortunate to have ChainCubed as our technology partner who provided us a blockchain-based voting platform. The beauty of this blockchain-based platform is that it’s open and transparent, but at the same time anonymous, secured, and “un-hackable.” As judges, we can all see other judges votes and comments anonymously.
After all the entries were scored, ChainCubed also went out and scraped the number of Likes each video entry received at the time when the data was compiled. This became the basis of the popular vote. Then we ranked ordered the video within each category based on the number of Likes the video received to normalize this metric.
All the judge scores were also collected from the ChainCubed voting platform. I have to manually exclude all the judges’ scores for entries that were marked as “conflict of interest.” If you recall our full disclosure honor system, the judges should self-declare any conflict of interest that could bias their scoring for a particular entry. Those scores must be excluded, otherwise, the entry would get a score of 0 (zero) from that judges. We obviously don’t want that. After this exclusion process, all the judges’ scores were ranked to normalize for the individual difference in each judges' scoring scale.
After the application of the rank statistics, both the popular vote (Likes) and the judges' vote should be normalized to the same scale, so they can be meaningfully combined. We computed a weighted average between the popular rank (based on the YouTube Likes) and each judge’s rank (based on the judge’s score). We gave the popular votes a weight of 2.5. That means the popular vote actually has a pretty significant effect on the final score. Specifically, it is equivalent to 2.5 judges. The final scores were rank again and the top 3 (or 2 if the category has only 3 entries) within each category were select as the finalist.
The Magic + Finalists
After all the hard work, here is where the fun begins. Remember from my 2nd blog post about OGA that as judges, we each have a magic booster, which we can apply to any entry across all the categories that we felt should be included in the finalists. Of the 7 judges, 3 of us have chosen to use our magic booster. I won’t speak for the other judges as to why they’ve chosen to boost a particular entry. They can do it for any reason they want. But since I also used my magic booster, I can certainly provide a bit more transparency to explain my choice (see below).
At last, here are the finalists for each category in no particular order:
Outstanding Gamification for Inclusion and Diversity sponsored by Gamification Nation
- Culture Shock
- Games for Peace - Arabia Felix in Yemen — magic boosted by me. (This entry was originally disqualified by me because it’s video was longer than 3 minutes. However, all the other judges scored it and gave it very positive comments that were visible (anonymized of course) on ChainCubed voting platform. I can’t help but give it a second look. And I’m glad I did, because I found it well deserves the finalist position as other judges. So I’ve decided to use my magic booster to boost this entry into the finalist shortlist.)
Outstanding Gamification Project in Audience Engagement sponsored by 3radical
- Siemens Revolutionises Selection Process with GBAs
- Gamification in Online Education
- The Hunt — magic boosted by GustavoT.
Outstanding Gamification Project in Learning sponsored by Growth Engineering
- Roots Gamification Impact
- Think Codex Customer Service Training Simulation
- GrandVision OPTIconic Sales Game
Outstanding Gamification Research sponsored by PlayVisit
- Gamification Incentives in VLE
- ScienceAtHome - Do Leaderboards Work?
- Framework for Design of Personalized Gamification — magic boosted by AnC
Outstanding Gamification Rookie sponsored by SelfDRVN
Outstanding Gamification Software sponsored by ChainCubed
Your Golden Ticket
Now that the finalists are announced, the ball is back in your court. As finalists, you all get a golden ticket. The golden ticket offers you many options to play and win (see my earlier post for detail):
- Second Chance: get coaching from a judge and resubmit your video, or
- Quantum Leap: jump 1 rank in a single component of your choice
- Do Nothing.
However, since we needed some lead time to produce the OGA trophy via 3D-printing, we are NOT offering the Second Chance option this time. However, you can still use the Quantum Leap option and choose which component (i.e. either impact, creativity or design) of your final rank you like to have a single-rank jump. All the finalists should have received an email congratulating your entry to the finalist round. That email should also ask you which component of the final score you want a quantum leap.
IMPORTANT: You must reply to that email and let us know by Friday, Nov 16, how you want to use your Quantum Leap. If we do not receive this reply from you by Nov 16, it’s assumed that your choice is to Do Nothing with your golden ticket.
So let us know how will you use your Golden Ticket by Friday, Nov 16.
Regardless of whether you win or not, I look forward to seeing all of you in Amsterdam. And if you are planning to be there, don’t miss my closing keynote on Do Androids Play Games, where we’ll explore gamification’s role in an imminent and inevitable future of robots and artificial intelligence (AI).