New crediting system

Message boards : Current tests : New crediting system

To post messages, you must log in.

Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 13 · Next

AuthorMessage
FluffyChicken

Send message
Joined: 17 Feb 06
Posts: 54
Credit: 710
RAC: 0
Message 2088 - Posted: 16 Aug 2006, 9:22:01 UTC - in response to Message 2086.  

Right. My fault for not using the right terminology. So we're on the same page now? Does this mean you agree or disagree with my points?
Basically I'm agree with Ethan's viewpoint.


What does that mean in plain english? Could you be more vague?

My point in this part of the discussion (that you objected to) was:

"In fact, I am a *huge* fan of the way SETI is doing it. And optomized clients mean exactly that...they crunch the SAME job in less time. More work = more credit = more science! Who can argue with that?"

So replace OC with SA to correct my language isssue. Do you still object?






Yes an optimised science application does more work in a given time (else it's badly optimised ;-)
BUT what has that got to do with what we are talking about here.

We are talking about people who either compile their own Boinc 'core client' aka 'client' or use a pre-compiled version that scores differently to the client given by boinc on their download page. (note: Everyone is free to do this, and the boinc site even points to some of them and gives instructions how to do this).

This is what the whole cofuffle is about. One one person score (claims credit for) for 12hrs work on a computer is often wildely different to what someone else may calim for that exact same computer if they use a different core client (akak crunch3rs, boinc studio, Mac superbench, truxoft or their own compiled not using 'boinc developers' settings)

Note, no mork science work is done just the credit someone has claimed.

You are talking about somethng completely different and irrelevant to this problem.

Probably just the usual boinc naming convention mix up :-)





Me, I see no reason to waste time retrospectively adjusting the credits.
(and yes mine are a mix of Crunch3rs, Trux, boincStudio, official and develoment. I don't even know what most are running at the moment though they'll probably be 5.4.9 / crunc3rs 5.5.0 / dev5.5.6/9 when I was trying to get the benchmarks differences.)

The only thing you do is please a minority of people here that are trying to help fix this, but you will piss off a whole bunch of people. NOT something you want to do given that is the majority of the enthusiasts and the top teams!
They have technically done nothing wrong.
(also do you keep adjusting if you find this new method does not work out)


Just a quick question, we are not going to have this split system (work credit and credit) at Rosetta?
I hope not as it just complicates matters. Just make sure it works over here at Ralph first with actually real measurements for a while through a variety of task.

Though be sure to set aside any other work to fix and adjust it as results come back in from Rosetta. There will be people with calclators checking them all (and good on them for making the checks!)
I'd say if youcan get a +-10% spread your doing very well and I'd leave it at that.
ID: 2088 · Report as offensive    Reply Quote
Ringold

Send message
Joined: 13 Aug 06
Posts: 2
Credit: 26,104
RAC: 0
Message 2090 - Posted: 16 Aug 2006, 10:12:31 UTC - in response to Message 2085.  

and some of the Intel P4s that are getting rather poor scores on the standard boinc benchmark when compared to AMD Athlon cpus.


Linux I don't know about, but keep in mind mhz for mhz, a P4 will bite the dust every time in comparison to an A64. Its architecture is just fundamentally inferior.

If boinc benchmarks are significantly different than the difference in other real, commonly accepted benchmarks (like those done at Anandtech, Bit-Tech, and HardOCP) then thats one thing. Otherwise, thats what one gets for buying a P4. Core 2 Duo, likewise, should womp Athlon 64 X2's. But considering an A64 3200+ isn't truly necessarily the equivalent of a P4 3.2ghz, as long as P4s generally trail roughly equivalent A64 chips then the benchmarks are actually right-on.

I can see why some of you guys see going back and updating the scores as a bad idea, stealing credits, etc. As far as you guys knew, and apparently with the admins blessing, cheating credits-wise was legal.

On the other hand, there's a ton of people who see the cheating going on, look at the rules and see no rule against it, and.. flame wars like this start up.

At the very least moving forward, I'm glad the system is changing. Going backwards, got to say it doesn't matter much to me. To those that really, really want scores to be revised back to Feb, consider this. Cheaters RAC will drop to sane levels, right? Some will even become upset that they can no longer cheat and their RAC will slowly fade to 0. Given not too much time at all, they'll fall to a place in their ranks closer to where they deserve anyway.

If it takes too much effort to revise the old scores on part of the folks running the program I hope it just gets left as it is. Time will sort out the rest at the very least.
ID: 2090 · Report as offensive    Reply Quote
dcdc

Send message
Joined: 15 Aug 06
Posts: 26
Credit: 89,834
RAC: 0
Message 2091 - Posted: 16 Aug 2006, 10:24:51 UTC - in response to Message 2088.  
Last modified: 16 Aug 2006, 10:26:53 UTC

First off, I think it needs to be accepted that some people will stop crunching Rosetta because of this credit change, no matter how it's done. However, it is also the case that many people aren't currently crunching because of the credit system, and many aren't pushing what they can because of the lack of fair competition. Of course the drop-off will be quick and the gains will be gradual. However, if the situation isn't rectified then it will always be a contentious issue and I believe the project will suffer in the long run, so I'm sure it's the right thing to do for the project.

I agree entirely with Ethan, and my position on the whole 'optimised' boinc manager situation is that it isn't in breach of the rules, but I consider it poor sportsmanship. Taking it to its logical extreme, if I add a magic Cell board to my computer that boosts my computer's SSE performance by 50x, Rosetta won't benefit in the slightest, but I'll get 50x more credits if using an optimised client. The credits aren't alligned to the work done.

I'd like that to be an irellevant point on this topic, but I'm afraid its going to be the central theme.

I do, however, appreciate what Zombie67 has said about people joining with one set of crediting rules, and then these being changed retro-actively.

However, I can't sway myself from thinking that we have the opportunity to make the credit system fairly accurate based entirely on work done. I'm fairly sure everyone agrees credit should be based on work done. We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. If not, you won't. We know that's not the case at the moment, and I don't see how anyone can logically be against that.

The right thing to do for the project is to keep as much crunching power as possible long term.

Just a quick question, we are not going to have this split system (work credit and credit) at Rosetta?
I hope not as it just complicates matters. Just make sure it works over here at Ralph first with actually real measurements for a while through a variety of task.


I agree - I don't think the stats sites would appreciate running two credit systems and it would just add to confusion.
ID: 2091 · Report as offensive    Reply Quote
tralala

Send message
Joined: 12 Apr 06
Posts: 52
Credit: 15,257
RAC: 0
Message 2092 - Posted: 16 Aug 2006, 10:43:21 UTC - in response to Message 2091.  

In any case it is good to break the whole process into seperate steps:

1. New credit system as an alternative
2. Abandoning the old credit system
3. Correcting past credits according to the new system

Currently we are at step 1 which is already announced on the Rosetta-Homepage. I sincerely hope until tomorrow there will be a page online which explains the new credit system as feet1st did. In fact they can just copy and past it but they _really_ should do this in order to avoid dozens of posts with the same questions in the message boards.

Step 2 and 3 can be discussed and planned once the first step works as intended.

ID: 2092 · Report as offensive    Reply Quote
STE\/E

Send message
Joined: 16 Feb 06
Posts: 27
Credit: 2,214,911
RAC: 0
Message 2093 - Posted: 16 Aug 2006, 10:55:48 UTC
Last modified: 16 Aug 2006, 11:16:09 UTC

We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me.

dcdc
New member
Joined: Aug 15, 2006
Posts: 3
ID: 1699
Credit: 0
RAC: 0
---

I can see why it wouldn't matter if the Credits are Back Dated to somebody that has 0 Credit 0 RAC & at the moment doesn't even have a Computer attached to the Project ... 0_o

It doesn't matter to me one way or the other if the Credits are Reset because it's what I've come to expect from the BOINC Projects, thanks for the help now here's your slap in the face.

With data base loses, bad application releases, WU's Erring out for no apparent reason not just at this Project but across all the Projects I've lost an enormous amount of Credits already so whats another 50,000 to 100,000 or more lost Credits.

I'll be the first to admit to using optimized clients that inflate the credits, but the Project said nor did nothing to curtail that practice. Now like a few people have said already the rules are going to be changed & lets back date it.

Who stands to gain the most from this Back Dating, the people with 0 Credit & 0 RAC I would suspect ... As a side NOTE when the Ralph Project started it was made clear the Credits were not important, In fact the Mods even made that clear a few times, now all of a sudden apparently the Credits are important ... Go Figure

ID: 2093 · Report as offensive    Reply Quote
Profile Astro

Send message
Joined: 16 Feb 06
Posts: 141
Credit: 32,977
RAC: 0
Message 2094 - Posted: 16 Aug 2006, 11:08:03 UTC
Last modified: 16 Aug 2006, 11:11:43 UTC

Unless I'm mistaken this new credit system has developed in three stages. The 2 credits/model stage, some calculated credit method stage used until roughly 14 August, and now some calculated credit method with a correction factor after that one. If you want to compare the 2 CR/Model stage my chart is here for the same puters as below. All are from standard boinc clients. From what I see, this new system needs to go through yet another stage prior to release, as my credit is still "all over the map", although is is closer to cross project parity than previous stages. I've changed the colors between stages two and three for easier viewing.

ID: 2094 · Report as offensive    Reply Quote
Profile Astro

Send message
Joined: 16 Feb 06
Posts: 141
Credit: 32,977
RAC: 0
Message 2095 - Posted: 16 Aug 2006, 11:16:47 UTC
Last modified: 16 Aug 2006, 11:24:10 UTC

Poorboy, dcdc has many attached computers to Rosetta as of a month ago. Although, I agree that anyone with zero credit has little at stake in this decision, and that always makes it easier.

ID: 2095 · Report as offensive    Reply Quote
STE\/E

Send message
Joined: 16 Feb 06
Posts: 27
Credit: 2,214,911
RAC: 0
Message 2096 - Posted: 16 Aug 2006, 11:17:24 UTC
Last modified: 16 Aug 2006, 11:18:11 UTC

Mine are all over the place to Tony, I get 2 Credits for 1 WU & 10 for another ... I also know dcdc has Computers attached to the Rosetta Project ...
ID: 2096 · Report as offensive    Reply Quote
Profile Jack Shaftoe

Send message
Joined: 8 Aug 06
Posts: 13
Credit: 105,163
RAC: 0
Message 2097 - Posted: 16 Aug 2006, 11:20:28 UTC - in response to Message 2093.  
Last modified: 16 Aug 2006, 11:28:04 UTC

(Deleting post, misunderstood)
ID: 2097 · Report as offensive    Reply Quote
Profile Jack Shaftoe

Send message
Joined: 8 Aug 06
Posts: 13
Credit: 105,163
RAC: 0
Message 2098 - Posted: 16 Aug 2006, 11:22:45 UTC - in response to Message 2091.  

I can't sway myself from thinking that we have the opportunity to make the credit system fairly accurate based entirely on work done. I'm fairly sure everyone agrees credit should be based on work done. We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. If not, you won't. We know that's not the case at the moment, and I don't see how anyone can logically be against that.


Precisely, very well said.
ID: 2098 · Report as offensive    Reply Quote
dcdc

Send message
Joined: 15 Aug 06
Posts: 26
Credit: 89,834
RAC: 0
Message 2099 - Posted: 16 Aug 2006, 11:28:56 UTC - in response to Message 2093.  
Last modified: 16 Aug 2006, 11:35:09 UTC

We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me.

dcdc
New member
Joined: Aug 15, 2006
Posts: 3
ID: 1699
Credit: 0
RAC: 0
---

I can see why it wouldn't matter if the Credits are Back Dated to somebody that has 0 Credit 0 RAC & at the moment doesn't even have a Computer attached to the Project ... 0_o

...

Who stands to gain the most from this Back Dating, the people with 0 Credit & 0 RAC I would suspect ... As a side NOTE when the Ralph Project started it was made clear the Credits were not important, In fact the Mods even made that clear a few times, now all of a sudden apparently the Credits are important ... Go Figure


I have no Ralph credits but we're discussing Rosetta.
[edit]i think there's a misunderstanding here - we're (or at least I'm) not taking about back-dating Ralph credits - this is about Rosetta credits[/edit]


With data base loses, bad application releases, WU's Erring out for no apparent reason not just at this Project but across all the Projects I've lost an enormous amount of Credits already so whats another 50,000 to 100,000 or more lost Credits.

I'll be the first to admit to using optimized clients that inflate the credits, but the Project said nor did nothing to curtail that practice. Now like a few people have said already the rules are going to be changed & lets back date it.


I know it's annoying to loose credits for work done, and I don't know about any of the other projects, but I don't know of any lost any data on Rosetta. Also, I know it didn't happen from the start, but you should have been credited for any WU errors for the last four months or so.

If you're being given credit for the work done I wouldn't consider that 'losing' credits though. You'd get the right credits for the amount of work completed.
ID: 2099 · Report as offensive    Reply Quote
STE\/E

Send message
Joined: 16 Feb 06
Posts: 27
Credit: 2,214,911
RAC: 0
Message 2100 - Posted: 16 Aug 2006, 11:36:44 UTC

I have no Ralph credits but we're discussing Rosetta.
-----

From what I have gathered at the Rosetta Project there will be no Back Dating of the Credits & all the testing that is being done here with the WU's & Credit Revision is to be eventually for the Rosetta Project, so if theres going to be no Credit Back Dating there where the most blatant Cheating was going on then why should it be done here ...
ID: 2100 · Report as offensive    Reply Quote
Profile Jack Shaftoe

Send message
Joined: 8 Aug 06
Posts: 13
Credit: 105,163
RAC: 0
Message 2101 - Posted: 16 Aug 2006, 11:42:47 UTC - in response to Message 2100.  
Last modified: 16 Aug 2006, 11:43:18 UTC

From what I have gathered at the Rosetta Project there will be no Back Dating of the Credits & all the testing that is being done here with the WU's & Credit Revision is to be eventually for the Rosetta Project, so if theres going to be no Credit Back Dating there where the most blatant Cheating was going on then why should it be done here ...


It is being proposed here that ROSETTA credits get back dated to Feb and reflect work done. If you go back a couple posts you'll see this (he's talking about Rosetta):


dekim
Forum moderator
Project administrator
Project developer
Project scientist

I'm extracting the archived data and will see what I can do. What do other users think about trying to backdate the credits using the new work based system?
ID: 2101 · Report as offensive    Reply Quote
suguruhirahara

Send message
Joined: 5 Mar 06
Posts: 40
Credit: 11,320
RAC: 0
Message 2102 - Posted: 16 Aug 2006, 11:42:56 UTC - in response to Message 2094.  
Last modified: 16 Aug 2006, 11:46:58 UTC

Unless I'm mistaken this new credit system has developed in three stages. The 2 credits/model stage, some calculated credit method stage used until roughly 14 August, and now some calculated credit method with a correction factor after that one. If you want to compare the 2 CR/Model stage my chart is here for the same puters as below. All are from standard boinc clients. From what I see, this new system needs to go through yet another stage prior to release, as my credit is still "all over the map", although is is closer to cross project parity than previous stages. I've changed the colors between stages two and three for easier viewing.

Nice work, but could you explain the points which separated 2nd and 3rd stage in detail more?
It is being proposed here that ROSETTA credits get back dated to Feb and reflect work done.
Just to make sure, does it mean that credits won't be deleted at all, and eveluate all of the credits granted with the new method and award again?
Thanks for reading,

ID: 2102 · Report as offensive    Reply Quote
Profile Astro

Send message
Joined: 16 Feb 06
Posts: 141
Credit: 32,977
RAC: 0
Message 2103 - Posted: 16 Aug 2006, 11:44:53 UTC

I'm confused too, I thought they were discussing changing work done at Rosetta back to February to the "work done" system, not here. Maybe we need some clarification??? I don't know if that's eliminating the old sytem, or simply augmenting rosetta with the addition of "work done" credits and the displays that go with it, so that's there'd be two credit systems in play one showing the old rosetta credits and one using the new "work done" system. I just don't know.
ID: 2103 · Report as offensive    Reply Quote
STE\/E

Send message
Joined: 16 Feb 06
Posts: 27
Credit: 2,214,911
RAC: 0
Message 2104 - Posted: 16 Aug 2006, 11:46:48 UTC
Last modified: 16 Aug 2006, 11:49:46 UTC

I'm extracting the archived data and will see what I can do. What do other users think about trying to backdate the credits using the new work based system?
----

It is my misunderstanding then if dekim's post was meant to be for the Rosetta Project, he wasn't specific about which Project he was referring to & I just took it as the Ralph Project ... As I haven't processed that much over @ the Rosetta Project for awhile I can't wait for the Fire Storm to hit the Project when & if it's done, I don't think the Dev's know what their letting themselves get into if thats implemented ... ;)
ID: 2104 · Report as offensive    Reply Quote
Profile Astro

Send message
Joined: 16 Feb 06
Posts: 141
Credit: 32,977
RAC: 0
Message 2105 - Posted: 16 Aug 2006, 11:56:40 UTC - in response to Message 2102.  

Nice work, but could you explain the points which separated 2nd and 3rd stage in detail more?

Thanks for reading,

I don't understand your question. I used black to represent stage two, and deep red to show results credited under stage three (if it exists). There appears to be a different credit system which came into play on/about August 14/15, though I'm not sure. If there is a stage three than it appears to be closer to project parity in terms of granted credit/hour, but values from my limited sample shows them all over the map still.

It's possible that with what they have, this is the best they can do. I just don't know. If it's the best then so be it, but if there's room for further improvement, then let's try out stage 4. lol For example, I know from my cross project analysis that for users of standard boinc clients, that granted credit is nearly the same (plus/minus 2 granted credits/hour) for a given system. To achieve project parity with results that vary, I would think as returned results which exceed "claimed credit" should be offset by the same number of results which are under the "claimed credit" value so that when they're averaged out come close to "claimed credit". If claimed credit for a machine is 10 credits/hour, then results should look like the following for one hours work:
10
10
7
13
5
15
9
11

so that they average out to 10. What I'm seeing now is that they average out to something higher than "claimed credit". Of course my sample has a small number of points and they should have a better "big picture" than I see.

ID: 2105 · Report as offensive    Reply Quote
dcdc

Send message
Joined: 15 Aug 06
Posts: 26
Credit: 89,834
RAC: 0
Message 2106 - Posted: 16 Aug 2006, 12:01:20 UTC - in response to Message 2104.  
Last modified: 16 Aug 2006, 12:04:21 UTC

As I haven't processed that much over @ the Rosetta Project for awhile I can't wait for the Fire Storm to hit the Project when & if it's done, I don't think the Dev's know what their letting themselves get into if thats implemented ... ;)


I think you're probably right. The most vocal group might be the winners on this, but as FC says, this might be the best result for the project if fewer leave.

I'd like to see what the effect on the data would be though as I expect it'd bring the top teams a bit closer together which'd be great for competition!
ID: 2106 · Report as offensive    Reply Quote
Profile Trog Dog
Avatar

Send message
Joined: 8 Aug 06
Posts: 38
Credit: 41,996
RAC: 0
Message 2107 - Posted: 16 Aug 2006, 12:03:01 UTC - in response to Message 2048.  


feet1st and Trog Dog:

We should first see what the response to the new method is and whether users would prefer it over the old. Not everyone will be happy about the final decision. I prefer abandoning the old system but that's just my opinion. Users that prefer the old system cannot use as an argument that they get more credits for whatever reason with the old system. The goal to even the playing field.


Cheers dekim

Thank you for your comments.

I have 15 boxes, hardly the most awe inspiring collection, but I'm constantly adding more and working on updating them, and they will all attach to Rosetta with a high resource share with the move to the new credit system.
ID: 2107 · Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 15 Feb 06
Posts: 20
Credit: 17,624
RAC: 0
Message 2108 - Posted: 16 Aug 2006, 12:09:51 UTC
Last modified: 16 Aug 2006, 12:14:09 UTC

But some people use the optimized client to compensate some low credits due to this or that. So a new credit system and a credit recalulation from February shouldn't change their results that much. They would be happy to have finally the real credits for the work done......

ID: 2108 · Report as offensive    Reply Quote
Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 13 · Next

Message boards : Current tests : New crediting system



©2024 University of Washington
http://www.bakerlab.org