New rating system - suggested values

Post by **gm_al** » Wed Jan 07, 2004 2:25 pm

As you have hopefully seen in the other thread, we are thinking to implement some rating system besides our VPs.

I call the rating system K-Value or short KVAL

== Rules == >>>(EDITED)<<<

There will be a rating value for every type of game GMs provide - currently WOK4 and WOK5, with more coming up as they are put into action (KVAL4, KVAL5 etc.)

Ratings are tracked for all types of WOK games run, including even the strangest X-games variations.

Every Player starts with a 1.000 rating in every game type. The notation is similar to the army level in WOK4/5.

There will be a "composite rating" (CKVAL) for each Player made out of all his/her individual ratings.

A rating can fall under 1.000 (ie. 0.955). The minimum is 0.000, the maximum is 9.999 (read - Egbert

)

Ratings are not timed or limited and are kept as long as a Player is active within Kaomaris. If a Player comes back after quitting the community, he re-starts with his initial (old) ratings.

Ratings and the number of each games played per game type will be tracked by the new KaoBase V2 - until it goes live, the GMs are asked to provide the Gatekeeper with all the informations needed.

Rating changes for WOK4/5:

- sending in orders: +0.001
- missing a Turn: -0.003
- going M-3: additional -0.010
- RIPping another Player: +0.010
- getting RIPped: -0.010
- every 0.5 VP awarded: +0.020

If a Player RIPs someone that went M-3 there will NOT be any rating changes ! Its either M-3 or RIP !

All WOK games that start on or after 1. January 2004 will be rated.

Rating changes are collected by the GMs on their Group pages, and transferred once the game officially ends.

==========

Open for discussion ! Lets try to get things rolling within a few days please, so we can see how it goes !

I would like to be able to track all games in 2004 !!

Saladin · Post by **Saladin** » Wed Jan 07, 2004 2:46 pm

Looks good, though i would prefer:

missing a Turn: -0.003

going M-3: additional -0.010

Brykovian · Post by **Brykovian** » Wed Jan 07, 2004 3:00 pm

What about the cases where a player get obliterated early, but not RIP'd ... a lot of times players will simply tell the GM, "Well, I'm done ... I won't be sending in any more orders."

Should we penalize someone like that for 3 missed turns, plus the M-3 double-jeopardy hit? (I say no -- score it like a RIP.)

-Bryk

Post by **gm_al** » Wed Jan 07, 2004 3:37 pm

Sending in no more orders AND SAYING SO TO THE GM means in fact sending in zero orders voluntarily. Thats a valid move as far as I can tell, so as long as the Player is still alive he will continue to get the +0.001 rating per Turn.

But then someone that RIPs him gets the +0.010 bonus, so chances are he wont be around for long....

I can see the -0.003 for missing Turns, and the additional -0.010 for going M-3 seems also fair. It means going M-3 in one move costs you 0.003+0.003+0.003+0.010 = 0.019 rating, which is about what 0.5 Vps are worth - that sounds good to me.

We're getting there !

Added: if a Player RIPs someone that went M-3 there will NOT be any rating changes ! Its either M-3 or RIP !

Ive edited the original post to reflect the discussion.

PS: GMs can start to add a "+/- KVAL" column to their waitlists....

Post by **Donut** » Wed Jan 07, 2004 5:03 pm

Hmm.... I wonder how often we'll see players hang around for several turns after being annhilated(sp?). I think this rating system could cause more aggressiveness in games. Maybe even shorten games.

I'll start keeping track of KVAL in my game. I'm still curious as to who to send the final tally to. Funty and (oh crap I forgot, I think Raw) said they would do it. But Al mentioned the gatekeeper...

Donut
GM Josh

Bjorn · Post by **Bjorn** » Wed Jan 07, 2004 5:53 pm

I would strongly suggest that each GM add a column to the group page that tracks the ratings points awarded during the game. When the game ends, the GM then informs the gatekeeper (me) who would then record the ratings points for each player by just looking at the group page.

If GMs did this then anyone with the time and inclination could record the ratings points. There is probably some way that the Kaobase 2 could somehow track this information. Mullog and Al will have to figure that out.

As far as the proposed points system goes, I have nothing to offer. I have always been a proponent of the ilk that winning is what matters. I am not fond of any system that rewards players for just 'hanging around'.

I know Al likes a ratings system where the average value increases over time. That is what the proposed system will do. I prefer rating systems that has a more or less constant average that player ratings move above or below. In a couple of years you will have new players with low ratings who are much better than higher rated players who have just played in more games.

Maybe we need to maintain a number that is the average ratings of all players in the data base and any new player starts off with the average rating instead of 1.000?

Post by **Donut** » Wed Jan 07, 2004 6:31 pm

Bjorn wrote: I know Al likes a ratings system where the average value increases over time. That is what the proposed system will do. I prefer rating systems that has a more or less constant average that player ratings move above or below. In a couple of years you will have new players with low ratings who are much better than higher rated players who have just played in more games.

Maybe we need to maintain a number that is the average ratings of all players in the data base and any new player starts off with the average rating instead of 1.000?

That's an interesting point. I think you're right. Over time, all players will accumulate an extremely high rating by just sending in orders. It also seems to me that sending in orders, or missing orders, doesn't apply to how good a player is. I think that in order to have a system where the average remains somewhat constant, it will have to be a system of Every point award has a point taken away. ie: You get a 2 points for RIPing someone and lose 2 points for getting RIPed.

But, if that is the case, how many things can we award points for? Maybe say that players that last 10 turns get 0 points but losing earlier lose a point per turn below 10; and vice versa.(10 is a random number, It should be somewhere in the middle of how many turns most games last).

I guess it'll just take looking at the best players, and then the not so best players, and looking to see what makes them different. I think Bjorn really has a point though in that the current idea for scoring would just lead to extremely high ratings for players down the road.

Donut

Brykovian · Post by **Brykovian** » Wed Jan 07, 2004 6:56 pm

I think Bjorn makes a great point ... not only does it favor players who have been around longer, but it greatly favors people who play a lot of games (as opposed to favoring players who have done well in the games they've played -- no matter how many they have played in).

Another thing is that encouraging players to hang around is fine, and even to play in a lot of games is fine, as long as there are games to be played. The biggest problem with WOK4/5 (as has been pointed out) is that there are not enough games being run. I'm not exactly sure how that would improve -- I doubt that adding points to a GM's player-rating for each game they complete would be a direction we'd want to go.

I would like to offer up using a tried-and-true system that already exists, such as the "Avalon Hill Reliability, Experience, and Ability Scoring System" (simply called "AREA" -- here's details on how it's calc'd: http://wolff.to/area/calcexp.html#_) ... I have a feeling that Bjorn might be familiar with that one.

In a nutshell, it works like this: Each player starts at 5000 and can gain or lose up to 200 points per game. Their gain/loss depends on the ratings of the players they beat or lost to. With a multi-player game, you need to find a way to rank-order the players as if they were in a race. For example, on a shared win, you'd have 2 #1 players, then the highest scoring surviving player would be #3, etc., until you get down to the first player knocked out of the game at #10. (If more than 1 player would get knocked out on the same round, they could tie at that position.) Then, you gain points for everyone below you and lose points for everyone above you. If two players tie then the higher-ranked player loses some points and the lower-ranked player gains some.

The math can get tricky ... but not too tricky for a good programmer (like mullog!!

) ... and I could whip together a quick VB app for anyone who'd want to do the tracking.

However ... if we go with the system that Al has proposed, then this comment is rather moot, now isn't it ...

-Bryk

Post by **gm_al** » Wed Jan 07, 2004 7:01 pm

Its interesting u seem to know already what will happen with the ratings over the years....

How about we try it out for a year and then talk about it again ?

Let me just tell you that no matter what system you implement there will always be pros and cons. Lets keep it simple for now, shall we ?

If you are affraid that over time it will 'bend' into one direction, I could suggest a decay value, ie.
- every year your rating decreases by 10% value if over 1.000 and increases by 10% if under 1.000

Or similar - LETS JUST TRY IT.

Brykovian · Post by **Brykovian** » Wed Jan 07, 2004 7:21 pm

gm_al wrote:Its interesting u seem to know already what will happen with the ratings over the years....

Well ... it's called math ...

How about we try it out for a year and then talk about it again ?

I'm game for that. Just trying to make a suggestion ... please don't feel offended, Al. Thought I saw a nice shiney, working wheel there and wondered by we'd go about reinventing it.

-Bryk

Saladin · Post by **Saladin** » Wed Jan 07, 2004 11:01 pm

As i initially was fully in favour of the rating system proposed by Al, i now however do feel that it should become an average so by simply having the amount divided by the number of games played. Otherwise the whole idea would be rather pointless as it would not really show the strength of a player but more how many games the player has played.

Bryk's idea of course is even better as it takes in to account WHO you've played, which of course makes all the difference!

BigJOzzy · Post by **BigJOzzy** » Thu Jan 08, 2004 12:29 am

Now lets take a look at this......winning a game will still be the most important part, lets look. I will use 20 turn games for the example just to make it simple

Wok 4
1VP (typically) so .04
20 tunrs so .02
RIP 3 players so .03

Total .09

Now other players that lasted until the end.
0 VP
20 turns so .02
RIP 2 players so .02
got RIP so -.01

Total .03

Now WOK 5 will even be more in favour of the victors.

On the other hand if you just play in a lot of games you will have to stay around in each game until at least turn 10 (sending in orders each turn) just to break even in the rating system. Then the only way to gain positive points is to RIP other players or make it more then 10 rounds. Therefore, it will show the better players by the use of the rating system.

Now one other thing will be that it will show the players that might RIP lots of player but aren't one of the last two players in the game. Which I think is a good thing, since at times players do all the work in RIPing other but then get taking out by the players that just sit and build.

Anyway those are just what come to me on first look. I think the system should prove to be useful to WOK, but only time will tell and we will have a better idea at this time next year.

Post by **gm_al** » Thu Jan 08, 2004 1:26 am

You gotta love that man - he rarely even posts, but when he does so it always makes sense !

Reminds me of that famous "best spying practice in WOK4" article in some old messenger issue....

Yeah, the system is quite balanced, and while its easy to claim "those who play a lot will gain more rating" (arent you tired to hear those "play a lot" arguments ?) it also means more EFFORT from that Players and never miss a Turn.

So two good things hopefully will happen:
- people miss less Turns
- people RIP other players more, making games faster and more exciting

Those two things alone would be an achievement.

PS: Sorry to say Sal, the main meaning is not about making averages out of games. We will still have the CKVAL to show off the best one in this regard.

cheers People !

Saladin · Post by **Saladin** » Thu Jan 08, 2004 9:03 am

gm_al wrote:PS: Sorry to say Sal, the main meaning is not about making averages out of games. We will still have the CKVAL to show off the best one in this regard.

Oh ok, so there will be an average per game shown as well, that's ok with me then.

Post by **korexus** » Thu Jan 08, 2004 1:46 pm

OK, just for fun (and to avoid work) I've had a quick look at the games I've played in since I joined. All these games were completed in the last scoring period so they would have been credited to this years' ranking if we'd started it last year. I haven't included games in which I am still playing, either of the clan champs or games which Seth abandoned.

Of the games which count I entered 13 and survived a total of 167 turns, I missed one turn and was RIPed 7 times. I didn't count up the number of people I RIPed as that would mean going through a lot of TRs...

However, even if I'd not RIPed anyone and then voted for someone else to win at the end of any games I survived, my K-VAL would have gone up by 0.094 - almost a tenth of it's original value. This does kind of suggest that the ratings will creep up over time. Also I think it would lead to more games ending in a vote. If you think you've lost it's only worth carrying on if you think you can last another 11 turns before dieing. Unless people will vote against themselves winning to try and get more RIPs under their belts.

I like the idea of the rating, but the numbers seem too arbitrary. If it's going to be calculated by Kaobase V2 anyway does it matter if the formula doesn't look very neat? - Tell you what, I can avoid more work by trying to work out something that doesn't just reward entering games.

korexus.

Post by **gm_al** » Thu Jan 08, 2004 1:52 pm

You forgot to mention that not everyone has your record in WOK playing.... how about we compare your virtual rating change to, say, CPs ?

You dont gain rating just by joining a game - you gain it by STAYING in it ACTIVELY.

Again, why not give it a try and see how things go ? We can always tweak the numbers later on (just need to keep all the records of the games for good)

Bjorn · Post by **Bjorn** » Thu Jan 08, 2004 2:12 pm

Keep the thoughts coming guys. I find this discussion most enlightening.

One thing I object to is simply awarding points for submitting moves and subtracting points for missing turns or going quit/M3. This has more to do with a players reliability than with ability. I have no objection to maintaining a reliability rating for players as well as an ability rating, but I don't believe the two should be confused. True, a player who sticks around for all 25 turns of a game is probably doing well, but that should be reflected by awarding ratings points for the final position, not for just getting there.

The original point made by Hannibal is the same old point that there are players who consistantly do well, but don't often win. The rating system should somehow reflect this. I like Raw's overall idea from the original thread quite a bit, but I also like Al's idea of rewarding players for RIPing others. How about the following.

Ratings points for a game are based on the order players are eliminated. In cases where two players share a win, the 1st and 2nd place points are pooled and split. All players still active in the game at the end will pool the points and split them. I would modify Raw's numbers a bit to reward a solo win a bit more.

+.10 for 1st
+.06 for 2nd
+.04 for 3rd and 4th
-.04 for 4th through 8th
-.05 for 9th and 10th

Note that this sums to 0, so the average will remain 1.000 over time.

In addition, I agree that a player should be awarded +.01 for RIPing an opponent and losing -.01 for being RIPed. If a player goes Quit/M3, then no points are awarded or lost for the elimination of that player.

So, in a shared win each of the winners would receive .08 points. If there are two survivors other than the winners then they will each get .04 points. If there are three survivors other than the winners, then they pool and split the points for 3-5 place (+.04 +.04 -.04)/3 so each of them would only get .013 points. (Certainly adds some incentive to kill off that 5th player.) Of course, each of them will get their own points for RIP other players as well.

In the end the ratings points awarded and lost always sums to zero, which I feel is necessary to have a meaningful number over the long haul.

Raw · Post by **Raw** » Thu Jan 08, 2004 3:14 pm

Hey guys...this is looking good, BUT....this seems to put a lot more work on the GM's shoulders.

Thoughts?

Will this scare games/GM's away?

-Raw

Post by **gm_al** » Thu Jan 08, 2004 3:51 pm

Very good post, Bjorn, indeed.

I like your concept, although I dont agree that sending in orders/missing a Turn isnt an "achievement". Maybe Im looking too much to increase stability by rewarding a Player for staying active, but then I believe it might help the game overall.

Your rating system has its merit, and maybe some flaws....(sorry to say):
- it treats WOK4 and WOK5 the same way (VPs arent really taken into account)
- you get only +/-0.1 for RIPping, thats not really well weighted considering I lose -0.5 when ending my game in 9th position - RIPping bust be seen as a reward worth going for
- I have some general problem with that ranking 1st-10th. If three Players are ripped on the same Turn it might well be a case of the OoP order on who of the RIPped wins rating points and who loses some....

I like that the sum of all rating changes equal zerop, but do we really need to have that ??

It all comes down to what will help the game - at least thats my view on things. Encourage those that play for good, never miss Turns, go after their neighbors and stay until they are RIPped, even if it doesnt give them VPs in the end.

Good discussion, keep it coming !

Bjorn · Post by **Bjorn** » Thu Jan 08, 2004 4:49 pm

gm_al wrote: - it treats WOK4 and WOK5 the same way (VPs arent really taken into account)

VPs are taken into account for lots of reasons, but why do they have have to matter for this rating system? Why treat WOK4 and WOK5 differently? The plan is to maintain separate WOK4 and WOK 5 ratings and average them. My feeling is that a win in WOK4 should have the same impact on your rating as a win in WOK 5.

gm_al wrote:- you get only +/-0.01 for RIPping, thats not really well weighted considering I lose -0.05 when ending my game in 9th position - RIPping must be seen as a reward worth going for.

Actually, you will go -0.06 for 9th position because you will lose an additional .01 for being RIPed. One problem with the system is that if you finish in 9th place for your first 17 games your rating will fall below 0. I'm not sure how we deal with that except to just set 0 as a minimum level.

So, make RIPing +/- .02. Of course, now a 9th place finish will cost you -.7 rating points, but now if you finish in 5th and managed to RIP two players along the way you will actually come out almost even.

gm_al wrote:- I have some general problem with that ranking 1st-10th. If three Players are ripped on the same Turn it might well be a case of the OoP order on who of the RIPped wins rating points and who loses some....

Players RIPed on the same turn should pool and split points to avoid having the OOP influence this. If the 4th and 5th place finishers are eliminated on the same turn then the +.04 and -.04 would be pooled and neither of them would lose points for final position, although they would each lose points for being RIPed.

gm_al wrote: I like that the sum of all rating changes equal zero, but do we really need to have that ??

If you want the rating to make sense over a long period of time, yes. This is not a game where someone who has been playing for awhile has collected items or experience points that convey an advantage to them in how the game engine treats them. The long term veteran does not have any different die roll modifiers or game system advantages over a new comer other than their experience.

gm_al wrote:It all comes down to what will help the game - at least thats my view on things. Encourage those that play for good, never miss Turns, go after their neighbors and stay until they are RIPped, even if it doesnt give them VPs in the end.

Again, I have no problem with giving awards to players who play for good, never miss turns and stay until they are RIPed. Players who do so will often finish in the top positions and gain rating points. Players who do not and miss turns, especially in WOK 5, will quickly be eliminated. I don't believe that ability points should be awarded for just sending in orders each turn and staying to the end. If you are unlucky or several people gang up on you to send you to an early exit, then you are going to lose rating points.

Note that the point scale I proposed is just that, a proposal. As long as the total sums to 0, then it works. The difficulty most people have with a zero sum system is that any attempt to increase the awards for good play must be paid for by those who played poorly or were unfortunate.