Posts by tralala

1) Message boards : RALPH@home bug list : minirosetta v1.48-1.51 bug thread (Message 4507)
Posted 23 Jan 2009 by tralala
Post:
Mtyka,

can you disable graphics for some WUs and see if the error rate goes down? Might help turning c)'s into b)'s.
2) Message boards : RALPH@home bug list : Bug reports for 5.56-5.59 (Message 2944)
Posted 29 Mar 2007 by tralala
Post:
Got a strange "Maximum Disk usage exceeded2 error. Runtime was 4 hours and Disk has 70 GB free, BOINC is allowed up to 50 GB and 99% disk usage. I guess the output file was too big or some other WU-specific settings were not correct.

http://ralph.bakerlab.org/result.php?resultid=475000


I guess stdout.txt reached the 100MB limit. I have a similar WU now and stdout ist at 50MB after two hours.
3) Message boards : RALPH@home bug list : Bug reports for 5.56-5.59 (Message 2941)
Posted 29 Mar 2007 by tralala
Post:
Got a strange "Maximum Disk usage exceeded2 error. Runtime was 4 hours and Disk has 70 GB free, BOINC is allowed up to 50 GB and 99% disk usage. I guess the output file was too big or some other WU-specific settings were not correct.

http://ralph.bakerlab.org/result.php?resultid=475000
4) Message boards : Current tests : Are we testing anything over here at the moment ? (Message 2243)
Posted 26 Aug 2006 by tralala
Post:
Since the credit testing has been done, what's to do ?

I think we are testing new WU whether they are causing troubles. I found the following from a Bakerlab employee in a now hidden post:


I'll try to clear this up quickly and succintly - running a workunit on Ralph is currently done in the lab to demonstrate that the workunit will function across a variety of platforms. During CASP typically run in the neighborhood of 100-500 workunits on Ralph to test that the workunit's input files were valid, that the tasks assigned to Rosetta by the workunit would not cause obvious problems on any platforms, and that the workunit completed in a reasonable time. We use Rosetta@Home to run multiple copies of a workunit (my largest WU total for a single job was 1.5 million!) so that we can analyze a distribution of structures produced by Rosetta.
5) Message boards : Current tests : New crediting system (Message 2238)
Posted 26 Aug 2006 by tralala
Post:
I hope someone rectifies this soon, you should certainly not have an account deleted for posting public information. Yes hide the post by all means (if the mod was unsure and wanted to protect something until he could get hold of the Administrator/David Baker)
Bad, Bad ,Bad managment.
Especially as they're new mods, they certainly shouldn't be doing that!

David Kim deleted the account of Aaron Finney which was absolutely necessary. He first deleted a few flaimbaits of him, than Aaron started to post the email and phone number of Dr. Baker together with a call to complain to David Baker by phone. This post was also removed and after he repeated it, his account was deleted. It was long after David Kim wanted to go to bed and in order to avoid more damage while he was at sleep the account was deleted. Had the ban feature be available over on Rosetta he might decided to just ban him, but the deletion was certainly justified.

Guys, please don't trust all the accusations which are thrown towards the project management at the moment.
6) Message boards : Current tests : Strange WUs (Message 2228)
Posted 23 Aug 2006 by tralala
Post:
I think a checkpoint is created after a modell or decoy is finished. That's according to my observation everytime your progressbar jumps to a new value.
Actually they put in checkpoints within a model run. They checkpoint after the ab initio search is done and do some checkpoints in the final relax stage. It differs from WU to WU though.
7) Message boards : Current tests : Strange WUs (Message 2225)
Posted 23 Aug 2006 by tralala
Post:
Theirry, the "no finished file" message means the manager temporarily lost track of the daemon. The daemon (which actually controls the work) should have kept running. It's just reporting the manager (the part you see) lost contact.

tony


OK... and that's why the Calculation time drop from 32 minutes to 18 minutes?


No, I suppose that's because between 18 and 32 minutes runtime there was no checkpoint. Checkpoints occur every 5 - 30 minutes depending on the WU and the speed of your computer (on slow computers it might even take longer in rare cases).
8) Message boards : Current tests : New crediting system (Message 2183)
Posted 19 Aug 2006 by tralala
Post:
...

I agree, if we add encryption to rosetta, we could use the internal benchmark. It's something to consider.


You'd also need to encrypt the in-memory benchmark results as well as the upload and download, also obfuscate the actual benchmark code so that people can't follow what it's doing in a debugger and modify the values on the fly. (Remember if people can use a debugger, an out-of-process routine can do the same once the memory locations are known).

The recent debugging code (map files etc) would prove extremely useful to someone trying to do this, but aren't necessary.

The new system does the job, and should do it well after the bugs are ironed out, so why mess around with insecure ways of doing the same thing?


I think it is way easier to fake the numbers of decoys generated than to fake the internal benchmark. So the currently tested system is not fake-proof either. I think though that this is not necessary since there will be only a very few people who really put in some "criminal energy" to achieve more credits.

I think we need to implement a new credit system soon. I voted for thorough testing on RALPH and not testing on Rosetta until it was really mature. However since we did test it on Rosetta togehter with the disastrous backdating discussion there is no way back and it should be solved soon, which means the new credit system must be implemented.
9) Message boards : Current tests : New crediting system (Message 2133)
Posted 16 Aug 2006 by tralala
Post:
these are very small numbers. it should even out. there will always be differences even with the same work unit because of the random nature of the runs.


You call 70% small numbers? That are definitely not small numbers! 10-20% I would call acceptable (not worth the cherry picking).

If you can easily prevent cherry picking why not do it?


4 results - that is a small number.


Well, you've been warned. ;-) If you are not convinced it's a problem try it out. :-)

I'm not sure what you meant with "we can easily prevent cherry picking". How would you do that? The only way to prevent cherry picking is to grant very similar credits/hour for each WU and if this is indeed easily achieved why not do it?
10) Message boards : Current tests : New crediting system (Message 2129)
Posted 16 Aug 2006 by tralala
Post:

I have one constructive and easy proposal though . ;-) Limit the number of WU sent to a specific host to one at a time with a one hour interval between sending work.


That would also limit people that don't give a rip about credits from loading a normal 2 or 3 day cache with work. Some (especially dial up users) only connect occaisionally, and if they can't get WUs during that time, they will be idle for the next 2 or 3 days.


This is intended for RALPH only where loading a cache is not senseful and doing other projects inbetween is recommended (especially Rosetta).
11) Message boards : Current tests : New crediting system (Message 2128)
Posted 16 Aug 2006 by tralala
Post:
these are very small numbers. it should even out. there will always be differences even with the same work unit because of the random nature of the runs.


You call 70% small numbers? That are definitely not small numbers! 10-20% I would call acceptable (not worth the cherry picking).

If you can easily prevent cherry picking why not do it?
12) Message boards : Current tests : New crediting system (Message 2125)
Posted 16 Aug 2006 by tralala
Post:
With just 4 results returned I have a range between 12 and 19 credits/hour, which is a difference of 60%. As Fluffy has pointed out this WILL result in cherrypicking and it WILL result in constant complaining on the boards. It would average out if people weren't so mad at getting more credits than others. As fluffy also pointed out it is easy for teams and individuals to almost automate this.

I have one constructive and easy proposal though . ;-) Limit the number of WU sent to a specific host to one at a time with a one hour interval between sending work. This would result in a more even distribution of WU since no host can grab 20 WU of the same kind and inflate the average credit/model of such a WU with over- or underclaiming on all those results. I described it here how it is done and here is the BOINC user guide entry on that.

edit: "Good result" (get more of those WU) and "bad result" (abort those WU)

edit1: fluffy's results differ even more: Result 1, Result 2
13) Message boards : Current tests : New crediting system (Message 2114)
Posted 16 Aug 2006 by tralala
Post:
I agree with mmciastro that the credit system in its current shape is not yet ready for deployment. :-( I get quite different granted credits/hour which will be a source of endless dispute and lead to cherrypicking of "credit-rich" WUs on Rosetta from those who aim for most credits. I wonder why the granted credits for different WU vary so much. How big is your sampling dekim? Perhaps there is too much inequality in the distribution of different WU on RALPH and too much uncertainty about the reported results that it does not work. It seems some WU were processed only by overclaiming clients while others were only processed by underclaiming clients. That needs to be corrected before rolling it out on Rosetta, either by restricting the estimation to trustable hosts or by ensuring the same host pool for each WU and the same factor each hosts contributes to the average credit/model. The more I think about it the more I think the easiest way would be to select different trustable hosts and use their values.

Why inventing a new credit system to level the playing field, which does not level it?

14) Message boards : Current tests : New crediting system (Message 2092)
Posted 16 Aug 2006 by tralala
Post:
In any case it is good to break the whole process into seperate steps:

1. New credit system as an alternative
2. Abandoning the old credit system
3. Correcting past credits according to the new system

Currently we are at step 1 which is already announced on the Rosetta-Homepage. I sincerely hope until tomorrow there will be a page online which explains the new credit system as feet1st did. In fact they can just copy and past it but they _really_ should do this in order to avoid dozens of posts with the same questions in the message boards.

Step 2 and 3 can be discussed and planned once the first step works as intended.
15) Message boards : Current tests : New crediting system (Message 2087)
Posted 16 Aug 2006 by tralala
Post:
I'm extracting the archived data and will see what I can do. What do other users think about trying to backdate the credits using the new work based system?


I'm afraid there will be quite an uproar if you correct the credit for past work. It would be the fair thing to do but in the end it will result in bitter discussions and a loss of hosts. I support transition from the old to the new system as soon as it is accepted in the boards to level the playing field but wouldn't touch the past.


P.S.: Yes I'm using the cheating Boinc 5.5.0 but no I don't care for credits. I use it now for scientific reasons to check whether the new credit system will be immune against overclaiming hosts on RALPH.
16) Message boards : Current tests : New crediting system (Message 2022)
Posted 12 Aug 2006 by tralala
Post:
So far only one result from my host:

http://ralph.bakerlab.org/result.php?resultid=242756

In this case it seems granted credit is a bit high. I suppose a host with my specs (Athlon 64@2.4 GHZ = Athlon 64 3800+) would get around 14-24 credits but I got even >25. This would even top Einstein which grants currently the most credit/hour.
17) Message boards : Current tests : New crediting system (Message 2003)
Posted 12 Aug 2006 by tralala
Post:
Everyone keep in mind that the current standard boinc crediting system will still be used.

Also, minor modifications to the credit/model values will not make that much of a difference in the long run. The important thing to know is that given any credit/model value, users will be on a level playing field. I think we can all agree that this is the major drive/motivation for coming up with a new method. Making sure it closely matches the BOIINC credit values is not as important since we will still use the old system along with the new.


I think it is a smart idea to introduce the new crediting system to Rosetta first as an alternative. In the long run however you need to decide on one system, which can only be the credit/model system, since it will be much fairer. The exact transition can be determined later and indeed the credit/model scheme allows quick and smooth adjusting in any case. Nevertheless I suggest introducing this on Rosetta only after it has been seen in effect here on RALPH with real data not the fixed test-ration 2credits/model. If this is too complicated then roll it out on Rosetta but I strongly support the idea of feet1st of putting together a documentation, which should be placed prominently (News-Entry and Sticky in message board).
18) Message boards : Current tests : New crediting system (Message 1982)
Posted 12 Aug 2006 by tralala
Post:
does anyone object to rolling this out to Rosetta@home?


Oh yes I strongly object. Please don't hasten! First let's discuss the details of how to determine the credit/model. How many WU/Results you plan to receive before determing the credit/model? Then let's determine some real credit/model here on RALPH and then reissue the WU here with the new credit factor and check if anyone is pleased. Then we should discuss questions like removing the top and bottom X percent, "adjusting" the credit in order to match with other projects. Then make an announcement on the HOMEPAGE of Rosetta that you plan to switch and link to RALPH if anybody wants to see what is coming. Then after a week make the switch.

That's a slow approach but I think it does much less harm to Rosetta and the atmosphere in the message boards to quarrel over the new credit system BEFORE it gets deployed than after that.
19) Message boards : Current tests : New crediting system (Message 1937)
Posted 8 Aug 2006 by tralala
Post:
One thing one should keep in mind is that nothing which is reported from the clientside is trustable, not the benchmarks, not the cpu-type and speed - nothing! Anything can be edited and faked. So every scenario which relies on the accuracy of the data from (single) clients should be discarded. However on average there are more accurate and valid information than faked ones. So any mechanism which takes averages from big numbers should in fact work quite well - even the claimed credits which depend on the reported benchmarks. If the sampling is big enough (>100) no discarding of odd results might be necessary but in any case I like the idea of throwing out 10% of the highest standard deviation.

Ethan

Your idea of measuring the average time per model and compare it with a golden computer assumes that the computer pools on Rosetta and Ralph are similar, which is probably not true. I assume on Ralph there are on average faster computer than on Rosetta. This makes the idea imho unpractical.
20) Message boards : Current tests : New crediting system (Message 1925)
Posted 8 Aug 2006 by tralala
Post:
Okay here is one proposal:

Send each type of WU to at least 60 hosts. Discard the results with the lowest and highest 10% claimed credit and take the arithmetic average of the remaining hosts as fixed credit/model for production runs on Rosetta.


Next 20



©2020 University of Washington
http://www.bakerlab.org