I had sent an email to Graham Somers at BHG -- he's the Big Huge Community guy -- on the topic of Ratings Pages. He was nice enough to send this reply:

I'd say most or all of us here are in agreement with the community and I've definitely been reading all of the posts about the ratings page. We're looking into some options, including ratings decay. Would you prefer decay (nothing too drastic) or a solution like the one that you have on your site?

Cool. It's great to know they're listening and considering their options. So, we'll ponder a bit before we respond.

Props to BHG! Say what you want about the multiplayer code, but BHG has been just about the most fan-friendly developer I've ever been involved with. Even if they don't do anything to change the ratings, just getting a response is better than a lot of developers out there. Hats off to GS and the rest of the BHG crew. Keep up the good work!

UPDATE: Here's what I sent back to Graham

Graham: Thanks for your response. It's refreshing to see a developer play an active role in the community. It's rare to see. I'm not entirely sure what the "perfect" system would be, but here are my thoughts....

I think the bottom line for players is that they want to see who's currently playing the best. The current page doesn't satisfy that because many of the players who dominate the top of the page haven't played in almost 3 months. Also it appears that the rating system previously used was a bit different from the current one.

Part of the problem lies in the fact that my rating is dependent on the rating of my opposition. If I can only find players who are 1900 or so, then I will never achieve a lofty rating. So it's useful for me to see how I compare against the current crop of players. In that case, I want to see a rating list that shows the top active players.

Defining active then is one issue. You'd need to set the minimum number of games and define the period in question. It seems reasonable that you should require a small number of games (3-5 perhaps) per month. You don't want the number too large, because then it rewards constant play, not skill. We just want people out there mixing it up. Players who don't meet that minimum don't have their rating displayed on the active list. In order to get them to care, you make the active list the default view when checking the ratings page.

It's logical to assume that if a player is not playing the minimum number of games per month then they are not maintaining their current rating level. So some sort of ratings decay might be in order. From discussing this with folks, there does seem to be consensus. The important thing is to keep it reasonable to ensure casual gamers aren't adversely affected. Also, you might want to establish a "floor" below which a player could never go below merely by decay. To satisfy those would don't like decay, it might be a good idea to also display (but not sort by -- make that a separate option) the highest established rating a player has attained. Another alternative is a "Hall of Fame" page.

It's also good to only display established ratings (or a minimum number of games), or at least to identify those ratings which are not established, perhaps with an asterisk. A 10 or 20-game minimum before making the board might be a good idea. Players with less than the minimum could still "see" where they stand, but their names wouldn't be displayed.

One more thing: it would be really nice if we could see better individual stats. Like what nations does a particular player play most often. Or what nations are the most preferred. It would also be nice to see these stats specifically for the rated room. This is where the rubber meets the road. Is everyone playing Mayan or is that just a rumor?

On a technical note: If the system used to calculate ratings has changed at all, then you should probably 1) do a one-time reset of "old ratings" or 2) recalculate those old ratings using the new methodology. It doesn't seem fair to compare old rating with new ratings.

To summarize:

• Only show active players -- you decide what determines active (I did played 1 game in last month)


• Establish a minimum threshold for display -- you can decide that, too (I did 10 games)


• Use decay to estimate current skill levels of payers who've stopped playing -- 100 points for every month not active, again your choice


• Add the "highest attained rating" to the display for each player


• Add more detailed player statistics where possible

Of course, O4B is just one voice among many, so share your thoughts people.
posted by Tim at 5:34 AM