Message boards : Current tests : New crediting system
Previous · 1 . . . 5 · 6 · 7 · 8 · 9 · 10 · 11 . . . 13 · Next
Author | Message |
---|---|
FluffyChicken Send message Joined: 17 Feb 06 Posts: 54 Credit: 710 RAC: 0 |
Right. My fault for not using the right terminology. So we're on the same page now? Does this mean you agree or disagree with my points?Basically I'm agree with Ethan's viewpoint. Yes an optimised science application does more work in a given time (else it's badly optimised ;-) BUT what has that got to do with what we are talking about here. We are talking about people who either compile their own Boinc 'core client' aka 'client' or use a pre-compiled version that scores differently to the client given by boinc on their download page. (note: Everyone is free to do this, and the boinc site even points to some of them and gives instructions how to do this). This is what the whole cofuffle is about. One one person score (claims credit for) for 12hrs work on a computer is often wildely different to what someone else may calim for that exact same computer if they use a different core client (akak crunch3rs, boinc studio, Mac superbench, truxoft or their own compiled not using 'boinc developers' settings) Note, no mork science work is done just the credit someone has claimed. You are talking about somethng completely different and irrelevant to this problem. Probably just the usual boinc naming convention mix up :-) Me, I see no reason to waste time retrospectively adjusting the credits. (and yes mine are a mix of Crunch3rs, Trux, boincStudio, official and develoment. I don't even know what most are running at the moment though they'll probably be 5.4.9 / crunc3rs 5.5.0 / dev5.5.6/9 when I was trying to get the benchmarks differences.) The only thing you do is please a minority of people here that are trying to help fix this, but you will piss off a whole bunch of people. NOT something you want to do given that is the majority of the enthusiasts and the top teams! They have technically done nothing wrong. (also do you keep adjusting if you find this new method does not work out) Just a quick question, we are not going to have this split system (work credit and credit) at Rosetta? I hope not as it just complicates matters. Just make sure it works over here at Ralph first with actually real measurements for a while through a variety of task. Though be sure to set aside any other work to fix and adjust it as results come back in from Rosetta. There will be people with calclators checking them all (and good on them for making the checks!) I'd say if youcan get a +-10% spread your doing very well and I'd leave it at that. |
Ringold Send message Joined: 13 Aug 06 Posts: 2 Credit: 26,104 RAC: 0 |
and some of the Intel P4s that are getting rather poor scores on the standard boinc benchmark when compared to AMD Athlon cpus. Linux I don't know about, but keep in mind mhz for mhz, a P4 will bite the dust every time in comparison to an A64. Its architecture is just fundamentally inferior. If boinc benchmarks are significantly different than the difference in other real, commonly accepted benchmarks (like those done at Anandtech, Bit-Tech, and HardOCP) then thats one thing. Otherwise, thats what one gets for buying a P4. Core 2 Duo, likewise, should womp Athlon 64 X2's. But considering an A64 3200+ isn't truly necessarily the equivalent of a P4 3.2ghz, as long as P4s generally trail roughly equivalent A64 chips then the benchmarks are actually right-on. I can see why some of you guys see going back and updating the scores as a bad idea, stealing credits, etc. As far as you guys knew, and apparently with the admins blessing, cheating credits-wise was legal. On the other hand, there's a ton of people who see the cheating going on, look at the rules and see no rule against it, and.. flame wars like this start up. At the very least moving forward, I'm glad the system is changing. Going backwards, got to say it doesn't matter much to me. To those that really, really want scores to be revised back to Feb, consider this. Cheaters RAC will drop to sane levels, right? Some will even become upset that they can no longer cheat and their RAC will slowly fade to 0. Given not too much time at all, they'll fall to a place in their ranks closer to where they deserve anyway. If it takes too much effort to revise the old scores on part of the folks running the program I hope it just gets left as it is. Time will sort out the rest at the very least. |
dcdc Send message Joined: 15 Aug 06 Posts: 27 Credit: 90,652 RAC: 0 |
First off, I think it needs to be accepted that some people will stop crunching Rosetta because of this credit change, no matter how it's done. However, it is also the case that many people aren't currently crunching because of the credit system, and many aren't pushing what they can because of the lack of fair competition. Of course the drop-off will be quick and the gains will be gradual. However, if the situation isn't rectified then it will always be a contentious issue and I believe the project will suffer in the long run, so I'm sure it's the right thing to do for the project. I agree entirely with Ethan, and my position on the whole 'optimised' boinc manager situation is that it isn't in breach of the rules, but I consider it poor sportsmanship. Taking it to its logical extreme, if I add a magic Cell board to my computer that boosts my computer's SSE performance by 50x, Rosetta won't benefit in the slightest, but I'll get 50x more credits if using an optimised client. The credits aren't alligned to the work done. I'd like that to be an irellevant point on this topic, but I'm afraid its going to be the central theme. I do, however, appreciate what Zombie67 has said about people joining with one set of crediting rules, and then these being changed retro-actively. However, I can't sway myself from thinking that we have the opportunity to make the credit system fairly accurate based entirely on work done. I'm fairly sure everyone agrees credit should be based on work done. We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. If not, you won't. We know that's not the case at the moment, and I don't see how anyone can logically be against that. The right thing to do for the project is to keep as much crunching power as possible long term. Just a quick question, we are not going to have this split system (work credit and credit) at Rosetta? I agree - I don't think the stats sites would appreciate running two credit systems and it would just add to confusion. |
tralala Send message Joined: 12 Apr 06 Posts: 52 Credit: 15,257 RAC: 0 |
In any case it is good to break the whole process into seperate steps: 1. New credit system as an alternative 2. Abandoning the old credit system 3. Correcting past credits according to the new system Currently we are at step 1 which is already announced on the Rosetta-Homepage. I sincerely hope until tomorrow there will be a page online which explains the new credit system as feet1st did. In fact they can just copy and past it but they _really_ should do this in order to avoid dozens of posts with the same questions in the message boards. Step 2 and 3 can be discussed and planned once the first step works as intended. |
STE\/E Send message Joined: 16 Feb 06 Posts: 27 Credit: 2,226,442 RAC: 783 |
We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. dcdc New member Joined: Aug 15, 2006 Posts: 3 ID: 1699 Credit: 0 RAC: 0 --- I can see why it wouldn't matter if the Credits are Back Dated to somebody that has 0 Credit 0 RAC & at the moment doesn't even have a Computer attached to the Project ... 0_o It doesn't matter to me one way or the other if the Credits are Reset because it's what I've come to expect from the BOINC Projects, thanks for the help now here's your slap in the face. With data base loses, bad application releases, WU's Erring out for no apparent reason not just at this Project but across all the Projects I've lost an enormous amount of Credits already so whats another 50,000 to 100,000 or more lost Credits. I'll be the first to admit to using optimized clients that inflate the credits, but the Project said nor did nothing to curtail that practice. Now like a few people have said already the rules are going to be changed & lets back date it. Who stands to gain the most from this Back Dating, the people with 0 Credit & 0 RAC I would suspect ... As a side NOTE when the Ralph Project started it was made clear the Credits were not important, In fact the Mods even made that clear a few times, now all of a sudden apparently the Credits are important ... Go Figure |
Astro Send message Joined: 16 Feb 06 Posts: 141 Credit: 32,977 RAC: 0 |
Unless I'm mistaken this new credit system has developed in three stages. The 2 credits/model stage, some calculated credit method stage used until roughly 14 August, and now some calculated credit method with a correction factor after that one. If you want to compare the 2 CR/Model stage my chart is here for the same puters as below. All are from standard boinc clients. From what I see, this new system needs to go through yet another stage prior to release, as my credit is still "all over the map", although is is closer to cross project parity than previous stages. I've changed the colors between stages two and three for easier viewing. |
Astro Send message Joined: 16 Feb 06 Posts: 141 Credit: 32,977 RAC: 0 |
Poorboy, dcdc has many attached computers to Rosetta as of a month ago. Although, I agree that anyone with zero credit has little at stake in this decision, and that always makes it easier. |
STE\/E Send message Joined: 16 Feb 06 Posts: 27 Credit: 2,226,442 RAC: 783 |
Mine are all over the place to Tony, I get 2 Credits for 1 WU & 10 for another ... I also know dcdc has Computers attached to the Rosetta Project ... |
Jack Shaftoe Send message Joined: 8 Aug 06 Posts: 13 Credit: 105,163 RAC: 0 |
(Deleting post, misunderstood) |
Jack Shaftoe Send message Joined: 8 Aug 06 Posts: 13 Credit: 105,163 RAC: 0 |
I can't sway myself from thinking that we have the opportunity to make the credit system fairly accurate based entirely on work done. I'm fairly sure everyone agrees credit should be based on work done. We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. If not, you won't. We know that's not the case at the moment, and I don't see how anyone can logically be against that. Precisely, very well said. |
dcdc Send message Joined: 15 Aug 06 Posts: 27 Credit: 90,652 RAC: 0 |
We have the opportunity to back date this to make the credit system consistent and fair throughout. If you've crunched more than me, you'll have more credit than me. I have no Ralph credits but we're discussing Rosetta. [edit]i think there's a misunderstanding here - we're (or at least I'm) not taking about back-dating Ralph credits - this is about Rosetta credits[/edit]
I know it's annoying to loose credits for work done, and I don't know about any of the other projects, but I don't know of any lost any data on Rosetta. Also, I know it didn't happen from the start, but you should have been credited for any WU errors for the last four months or so. If you're being given credit for the work done I wouldn't consider that 'losing' credits though. You'd get the right credits for the amount of work completed. |
STE\/E Send message Joined: 16 Feb 06 Posts: 27 Credit: 2,226,442 RAC: 783 |
I have no Ralph credits but we're discussing Rosetta. ----- From what I have gathered at the Rosetta Project there will be no Back Dating of the Credits & all the testing that is being done here with the WU's & Credit Revision is to be eventually for the Rosetta Project, so if theres going to be no Credit Back Dating there where the most blatant Cheating was going on then why should it be done here ... |
Jack Shaftoe Send message Joined: 8 Aug 06 Posts: 13 Credit: 105,163 RAC: 0 |
From what I have gathered at the Rosetta Project there will be no Back Dating of the Credits & all the testing that is being done here with the WU's & Credit Revision is to be eventually for the Rosetta Project, so if theres going to be no Credit Back Dating there where the most blatant Cheating was going on then why should it be done here ... It is being proposed here that ROSETTA credits get back dated to Feb and reflect work done. If you go back a couple posts you'll see this (he's talking about Rosetta):
|
suguruhirahara Send message Joined: 5 Mar 06 Posts: 40 Credit: 11,320 RAC: 0 |
Unless I'm mistaken this new credit system has developed in three stages. The 2 credits/model stage, some calculated credit method stage used until roughly 14 August, and now some calculated credit method with a correction factor after that one. If you want to compare the 2 CR/Model stage my chart is here for the same puters as below. All are from standard boinc clients. From what I see, this new system needs to go through yet another stage prior to release, as my credit is still "all over the map", although is is closer to cross project parity than previous stages. I've changed the colors between stages two and three for easier viewing. Nice work, but could you explain the points which separated 2nd and 3rd stage in detail more? It is being proposed here that ROSETTA credits get back dated to Feb and reflect work done.Just to make sure, does it mean that credits won't be deleted at all, and eveluate all of the credits granted with the new method and award again? Thanks for reading, |
Astro Send message Joined: 16 Feb 06 Posts: 141 Credit: 32,977 RAC: 0 |
I'm confused too, I thought they were discussing changing work done at Rosetta back to February to the "work done" system, not here. Maybe we need some clarification??? I don't know if that's eliminating the old sytem, or simply augmenting rosetta with the addition of "work done" credits and the displays that go with it, so that's there'd be two credit systems in play one showing the old rosetta credits and one using the new "work done" system. I just don't know. |
STE\/E Send message Joined: 16 Feb 06 Posts: 27 Credit: 2,226,442 RAC: 783 |
I'm extracting the archived data and will see what I can do. What do other users think about trying to backdate the credits using the new work based system? ---- It is my misunderstanding then if dekim's post was meant to be for the Rosetta Project, he wasn't specific about which Project he was referring to & I just took it as the Ralph Project ... As I haven't processed that much over @ the Rosetta Project for awhile I can't wait for the Fire Storm to hit the Project when & if it's done, I don't think the Dev's know what their letting themselves get into if thats implemented ... ;) |
Astro Send message Joined: 16 Feb 06 Posts: 141 Credit: 32,977 RAC: 0 |
Nice work, but could you explain the points which separated 2nd and 3rd stage in detail more? I don't understand your question. I used black to represent stage two, and deep red to show results credited under stage three (if it exists). There appears to be a different credit system which came into play on/about August 14/15, though I'm not sure. If there is a stage three than it appears to be closer to project parity in terms of granted credit/hour, but values from my limited sample shows them all over the map still. It's possible that with what they have, this is the best they can do. I just don't know. If it's the best then so be it, but if there's room for further improvement, then let's try out stage 4. lol For example, I know from my cross project analysis that for users of standard boinc clients, that granted credit is nearly the same (plus/minus 2 granted credits/hour) for a given system. To achieve project parity with results that vary, I would think as returned results which exceed "claimed credit" should be offset by the same number of results which are under the "claimed credit" value so that when they're averaged out come close to "claimed credit". If claimed credit for a machine is 10 credits/hour, then results should look like the following for one hours work: 10 10 7 13 5 15 9 11 so that they average out to 10. What I'm seeing now is that they average out to something higher than "claimed credit". Of course my sample has a small number of points and they should have a better "big picture" than I see. |
dcdc Send message Joined: 15 Aug 06 Posts: 27 Credit: 90,652 RAC: 0 |
As I haven't processed that much over @ the Rosetta Project for awhile I can't wait for the Fire Storm to hit the Project when & if it's done, I don't think the Dev's know what their letting themselves get into if thats implemented ... ;) I think you're probably right. The most vocal group might be the winners on this, but as FC says, this might be the best result for the project if fewer leave. I'd like to see what the effect on the data would be though as I expect it'd bring the top teams a bit closer together which'd be great for competition! |
Trog Dog Send message Joined: 8 Aug 06 Posts: 38 Credit: 41,996 RAC: 0 |
Cheers dekim Thank you for your comments. I have 15 boxes, hardly the most awe inspiring collection, but I'm constantly adding more and working on updating them, and they will all attach to Rosetta with a high resource share with the move to the new credit system. |
[B^S] thierry@home Send message Joined: 15 Feb 06 Posts: 20 Credit: 17,624 RAC: 0 |
But some people use the optimized client to compensate some low credits due to this or that. So a new credit system and a credit recalulation from February shouldn't change their results that much. They would be happy to have finally the real credits for the work done...... |
Message boards :
Current tests :
New crediting system
©2024 University of Washington
http://www.bakerlab.org