I'll throw something else in here. One advantage of the proposed new credit system over any that use benchmarks is credit-crunching alignment, which is the very thing we've been requesting here. Benchmarks are rarely perfectly aligned with the work to be done. As one example, the size of CPU cache may play a large part in crunching ability, but it is rarely a factor that influences benchmarks as they tend to be small and quick.

By not using a benchmark, or rather using a whole decoy as a benchmark, the credits are exactly aligned with the credit system. The only issue remaining is that there is variation in time taken to produce decoys on any given machine, but no variation in credit awarded. If this averages accurately, then I believe it to be the best system. If someone tweaks their computer by, say increasing the FSB, which removes a CPU-RAM bottleneck, this is unlikely to be picked up in any benchmark, but will show up in this system.

The other option of course is to try and make the benchmark realistic. This is difficult, because of the size and variation in algorithms used within the WUs. The 'new credit system' includes for this.

My final point regards the golden machines used. I've posted somewhere - here or at Rosetta - that the makeup of the golden machines needs to reflect the wild population for the following reason:

If the golden machines are 10 AMDs running Windows, which are very good at 2 in 10 WUs (as they fit into a small cache), but take much longer with the remaining 8 (as they are slightly larger than the AMD cache) then the first 2 WUs will be assigned little credit, while the remaining 8 will be assigned more. In the wild, although a Pentium isn't affected by the difference between the WUs sizes due to its larger cache, it will still get the credits that reflect how the AMDs crunched them.

This may be a minor point with regard to CPU cache sizes (or it may not), but there are a great number of factors that could influence this, including whetstone performance, dhrystone performance, CPU cache size, FSB/HyperTransport speeds, RAM availability etc...

The place that matches the wild population of computers the most is, of course, the wild population! Thats my current take on a credit system.

I can see that a well aligned benchmark can be more accurate in the short term, which is good for those wanting to push their computers, but after a week or two of crunching I'd be suprised if this system didn't prove more accurate.

cheers
Danny

---

...

I agree, if we add encryption to rosetta, we could use the internal benchmark. It's something to consider.

You'd also need to encrypt the in-memory benchmark results as well as the upload and download, also obfuscate the actual benchmark code so that people can't follow what it's doing in a debugger and modify the values on the fly. (Remember if people can use a debugger, an out-of-process routine can do the same once the memory locations are known).

The recent debugging code (map files etc) would prove extremely useful to someone trying to do this, but aren't necessary.

The new system does the job, and should do it well after the bugs are ironed out, so why mess around with insecure ways of doing the same thing?

I think it is way easier to fake the numbers of decoys generated than to fake the internal benchmark. So the currently tested system is not fake-proof either. I think though that this is not necessary since there will be only a very few people who really put in some "criminal energy" to achieve more credits.

I think we need to implement a new credit system soon. I voted for thorough testing on RALPH and not testing on Rosetta until it was really mature. However since we did test it on Rosetta togehter with the disastrous backdating discussion there is no way back and it should be solved soon, which means the new credit system must be implemented.

---

I think it is way easier to fake the numbers of decoys generated than to fake the internal benchmark. So the currently tested system is not fake-proof either.

I'd be very suprised if that were true, but I'd like to hear DK's thoughts on the subject.

---

I think it is way easier to fake the numbers of decoys generated than to fake the internal benchmark. So the currently tested system is not fake-proof either.

I'd be very suprised if that were true, but I'd like to hear DK's thoughts on the subject.

A user can fake the [edit: claimed credits, not the internal benchmark] by just editing a file just as a user can fake the numbers of decoys generated. The difference is that we can validate the decoys and catch cheaters. Unless we start encrypting data or use redundancy, the system is not going to be fake proof.

---

Unless we start encrypting data ... the system is not going to be fake proof.

You sir, hold all the cards in regards to that.

---

A user can fake the [edit: claimed credits, not the internal benchmark] by just editing a file just as a user can fake the numbers of decoys generated. The difference is that we can validate the decoys and catch cheaters. Unless we start encrypting data or use redundancy, the system is not going to be fake proof.

No, it's not.

I have my last Ralph WU in my cache, and when that's returned, I'll move my Ralph share to uFluids. So this is goodbye from me.

I don't expect a personal thank you for my contribution from David Baker, let alone a thank you from him on my teamboard.

Goodbye.

---

[color=navy][b]"I'm trying to maintain a shred of dignity in this world." - Me[/b][/color]

A user can fake the [edit: claimed credits, not the internal benchmark] by just editing a file just as a user can fake the numbers of decoys generated. The difference is that we can validate the decoys and catch cheaters. Unless we start encrypting data or use redundancy, the system is not going to be fake proof.

No, it's not.

I have my last Ralph WU in my cache, and when that's returned, I'll move my Ralph share to uFluids. So this is goodbye from me.

I don't expect a personal thank you for my contribution from David Baker, let alone a thank you from him on my teamboard.

Goodbye.

Forgive me.. I know you're having a moment - but I'm confused, and I need your help to understand things - You're leaving because we're trying to figure out how to prevent falsification of credit data?

---

We can limit the number of wu you can abort. We can change the work unit distribution to be more homogenous. We can update the credit/model values. We can penalize people trying to cherry pick. It seems easy enough to me if it becomes a problem. just some ideas off the top of my head.

Well I hope you don't limit me for all the WU's I have been aborting recently in both Ralph and (mainly) Rosetta, due to WU's locking up and doing nothing (I have reported this a number of times but nothing has happened). Happens on Linux machines and needs rebooting to clear.
By the way if problems are occuring and WU's have to be aborted, as in my case due to WU lock ups, and Rosetta/Ralph admin limits what you can do with the WU's how can you keep working? And are we still going to get credit for errored work or units that fail for other reasons?

---

Forgive me.. I know you're having a moment - but I'm confused, and I need your help to understand things - You're leaving because we're trying to figure out how to prevent falsification of credit data?

I think Fuzzy is signing off from Ralph and Rosetta over David Bakers postings on the XS boards, and the disregard that this showed to all the Rosetta/Ralph participants.

---

Hi,

I am not sure if this of value for the crediting system, but I have done a check of some 20 Rosetta WUs and have found out that the time to calculate one decoy is pretty exactly proportional to the number of amino acids in the protein to the power of 1,3.

My formula is (number of amino acids)^1.3*n(decoys) / time = const.(for a given machine)

It yields pretty good values that vary by an average of 2.3% around the median. I am still collecting numbers to compare, but the latest two samples I put in after adjusting the proportionality factor had 99,9 and 100,2 of the average "work factor" for my machine. And the length of proteins varies from 28 to 157 amino acids, which is a factor of nearly six in length.

If that also accounts for other machines it would be pretty easy to just calculate a number of given WUs of classified types several times to get a group of average results that can be applied to all other machines out there. Then all you would need to grant credits for any other WU is the number of amino acids in the protein and the method used for the calculation. You would have to repeat the calibration as soon as you change the method, but as long as the method is unchanged the credit is predictable and can thus be directly be derived from the number of decoys computed.

Of course, the new system seems to get into the swing pretty well. But maybe that observation might help to implement a system that relies less on precalculating credits on Ralph machines but on an empirical formula that can be derived from the number of amino acids in the template and the specific mathematical method used in the WU?

Regards,

Christoph

[edited for clarity]

For anyone left that questions the need for some form of new credit system, I illustrait how easy it is to falsify the BOINC client credit claims. I think the ease with which it is achieved will be news to some.

One test for the new system would be for someone to publish similar step-by-step instructions on how to falsify results under the new system. At this point I believe it is still possible, but more difficult. And with a few more fairly minor changes (I've suggested encryption of small amounts of the result data which will be required for your WU to be authenticated by the Rosetta server), this can be thwarted as well.

---

I thought there were joining for science.

Blame Misfit!

---

me@rescam.org

I thought there were joining for science.

Wish it were true more then it appears to be.

---

I thought there were joining for science.

Wish it were true more then it appears to be.

Appearances can be deceiving.. My nephew handed me a blue plastic light saber, and picked up the red plastic light saber. I deflected his swings a few times, and then he pressed the red blade against the blue blade and I made a poor imitation of the crackling noise that is supposed to occur when the blades are in contact. The Irish Wolf hound in the house goes nuts and lunges at me for attacking my nephew, when in fact it's the little miscreant that's attacking and he's enjoying the activity.

Obviously not my favorite mutt since the attacks started; but he also freaks out and lets everyone know when diabetics in the house go into insulin shock. (His life has been saved due to his being an instant medical alert system..)

As for the credit system, here's hoping that the last few snags in the system get worked out, so we can get back to the fun of competitive crunching or reliable crunching - whichever of the two is the main justification for our person contributions here.

---

argh.. timed out, and it still posted the first time. :)

---

No, it's not.

I have my last Ralph WU in my cache, and when that's returned, I'll move my Ralph share to uFluids. So this is goodbye from me.

I don't expect a personal thank you for my contribution from David Baker, let alone a thank you from him on my teamboard.

Goodbye.

I left rosetta too. 28k credits a month I was giving to Rosetta.. now 0.

I tried vocally to argue against the credit system on the rosetta boards - BOY WAS THAT A MISTAKE.

The german mod deleted my account because I posted David's email and the # to wash U (both of which are on his webpage.) SORRY - DIDN'T KNOW POSTING PUBLIC INFO YOU ARE GIVING FREELY AWAY ANYWAY WAS AGAINST YOUR RULES.

Don't expect me to reattach.

I have also removed my endorsement for this project from the International AIDS Conference. It's clear that your moderators have lost control of the project for you.

---

I left rosetta too. 28k credits a month I was giving to Rosetta.. now 0.

I tried vocally to argue against the credit system on the rosetta boards - BOY WAS THAT A MISTAKE.

The german mod deleted my account because I posted David's email and the # to wash U (both of which are on his webpage.) SORRY - DIDN'T KNOW POSTING PUBLIC INFO YOU ARE GIVING FREELY AWAY ANYWAY WAS AGAINST YOUR RULES.

Don't expect me to reattach.

I have also removed my endorsement for this project from the International AIDS Conference. It's clear that your moderators have lost control of the project for you.

I hope someone rectifies this soon, you should certainly not have an account deleted for posting public information. Yes hide the post by all means (if the mod was unsure and wanted to protect something until he could get hold of the Administrator/David Baker)

Bad, Bad ,Bad managment.

Especially as they're new mods, they certainly shouldn't be doing that!

---

I hope someone rectifies this soon, you should certainly not have an account deleted for posting public information. Yes hide the post by all means (if the mod was unsure and wanted to protect something until he could get hold of the Administrator/David Baker)
Bad, Bad ,Bad managment.
Especially as they're new mods, they certainly shouldn't be doing that!

David Kim deleted the account of Aaron Finney which was absolutely necessary. He first deleted a few flaimbaits of him, than Aaron started to post the email and phone number of Dr. Baker together with a call to complain to David Baker by phone. This post was also removed and after he repeated it, his account was deleted. It was long after David Kim wanted to go to bed and in order to avoid more damage while he was at sleep the account was deleted. Had the ban feature be available over on Rosetta he might decided to just ban him, but the deletion was certainly justified.

Guys, please don't trust all the accusations which are thrown towards the project management at the moment.

---

I hope someone rectifies this soon, you should certainly not have an account deleted for posting public information. Yes hide the post by all means (if the mod was unsure and wanted to protect something until he could get hold of the Administrator/David Baker)
Bad, Bad ,Bad managment.
Especially as they're new mods, they certainly shouldn't be doing that!

David Kim deleted the account of Aaron Finney which was absolutely necessary. He first deleted a few flaimbaits of him, than Aaron started to post the email and phone number of Dr. Baker together with a call to complain to David Baker by phone. This post was also removed and after he repeated it, his account was deleted. It was long after David Kim wanted to go to bed and in order to avoid more damage while he was at sleep the account was deleted. Had the ban feature be available over on Rosetta he might decided to just ban him, but the deletion was certainly justified.

Guys, please don't trust all the accusations which are thrown towards the project management at the moment.

Repeatedly posting is bad especially when you've been told not to, it is rosetta board after all not ours. Though still I don't get the point of the ban, you can just sign back up.
I had also assumed they had implemented the ban feature (since they updated the boards (or was that only here).

---

This post was also removed and after he repeated it, his account was deleted.

BULL*&%.

I did NOT repeat the post. NO warning, No NOTHING. If it posted the exact same message twice then it's -ONLY- because your server had a hiccup and double posted the damn thing. It is absolutely unconscionable that you would use DELETION for posting publicly available information you can grab from U Of W. Also - Maybe if your german moderator wasn't deleting EVERY SINGLE NEGATIVE MESSAGE about the credit system, It wouldn't have led to this.

Your moderator over there overstepped his bounds in the first place. We should be able to comment as we feel appropriate regarding systemwide credit changes. IT SEEMS FITTING THAT THE MOD WAS FROM GERMANY, DOESN'T IT? The general idea of information control over there is appalling. It's a shame you've let this fool ruin your 'public' forum.

Forum is supposed to mean 'an assembly, meeting place, television program, etc., for the open discussion of questions of public interest.'

NOT - We're going to silence anything that sounds negative so that a little less &$%% hits the wall after going through the fan.

Repeatedly posting is bad especially when you've been told not to, it is rosetta board after all not ours. Though still I don't get the point of the ban, you can just sign back up.
I had also assumed they had implemented the ban feature (since they updated the boards (or was that only here).

Also.. Wasn't a ban. DELETION. Check my boincstats. You will see no Rosetta there. 90,000 for Rosetta, POOF, because I copied and pasted from www.washington.edu

F-ING BAD FORM FOR ROSETTA.

---