Request: Increase of deadline for Virtualbox-using rosetta python app

Message boards : Number crunching : Request: Increase of deadline for Virtualbox-using rosetta python app

To post messages, you must log in.

AuthorMessage
Profile Michael H.W. Weber
Avatar

Send message
Joined: 16 Feb 06
Posts: 2
Credit: 22,265
RAC: 0
Message 7471 - Posted: 6 Jan 2024, 13:11:27 UTC
Last modified: 6 Jan 2024, 13:15:52 UTC

Due to the immense disk space and RAM requirements, I would like to ask for an increase of the deadlines of the Virtualbox-using rosetta python tasks.

The base data to run these tasks require approx 11 GB on disk.
On top of that, each single tasks additionally generates around 15 GB of data.
In summary, running a single taks requires at least 26 GB of disk space and each additional work unit will add another 15 GB - with excessive disk I/O, by the way.

Given the fact that running ONLY two tasks of these in parallel will almost entirely occupy the RAM of a 16 GB Windows 10 system making additional multi-tabbed browsing virtually impossible, these tasks on many systems require to be run sequentially to avoid system crashes.

Given the deadline of currently 3 days with a single task taking about 6,5 hrs on an i5-2500K processor, I'd like to suggest a moderate increase in the return deadline for these specific WUs.
Do you think this would be possible?

Michael.
President of Rechenkraft.net
ID: 7471 · Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 9 Apr 08
Posts: 841
Credit: 1,888,960
RAC: 0
Message 7472 - Posted: 7 Jan 2024, 10:51:55 UTC - in response to Message 7471.  
Last modified: 7 Jan 2024, 10:53:06 UTC

Due to the immense disk space and RAM requirements, I would like to ask for an increase of the deadlines of the Virtualbox-using rosetta python tasks.
Given the deadline of currently 3 days with a single task taking about 6,5 hrs on an i5-2500K processor, I'd like to suggest a moderate increase in the return deadline for these specific WUs.
Do you think this would be possible?


+1

But i think they do NOT read the forum, so it's like a voice in the desert
I'll try to contact someone by Twitter/X o by mail....
ID: 7472 · Report as offensive    Reply Quote
Profile Michael H.W. Weber
Avatar

Send message
Joined: 16 Feb 06
Posts: 2
Credit: 22,265
RAC: 0
Message 7473 - Posted: 7 Jan 2024, 12:47:18 UTC - in response to Message 7472.  

But i think they do NOT read the forum, so it's like a voice in the desert
I'll try to contact someone by Twitter/X o by mail....

Thanks a lot.

Maybe I should also add that, since the rosetta python app runs in Virtualbox, it appears that BOINC does not have full control over the RAM usage?

Example:
When I set maximum RAM usage to 80% in the BOINC manager, the double rosetta python task situation certainly exceeds that.
It seems that BOINC just measures the Virtualbox wrapper's RAM consumption but not what is actually running within that virtual machine. That stuff isn't even listed among the BOINC-related tasks.
Just an idea...

Michael.
President of Rechenkraft.net
ID: 7473 · Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 9 Apr 08
Posts: 841
Credit: 1,888,960
RAC: 0
Message 7475 - Posted: 7 Jan 2024, 18:14:39 UTC - in response to Message 7473.  
Last modified: 7 Jan 2024, 18:14:54 UTC

Maybe I should also add that, since the rosetta python app runs in Virtualbox, it appears that BOINC does not have full control over the RAM usage?

You can control the use of ram in the vbox_job.xml file.
But it's released by the projects, so i think there is a motivation for the usage of ram.

Meantime, if the project update the wrapper is not a bad idea: now they are using 26203 (3 years old) and the newest is 26207 (December 2023)
Since 26204 there is the multi-attach functionality
ID: 7475 · Report as offensive    Reply Quote
Fardringle

Send message
Joined: 22 Feb 06
Posts: 6
Credit: 257,195
RAC: 0
Message 7476 - Posted: 8 Jan 2024, 5:11:57 UTC

Adding to the complexity of completing these tasks on time, most of the tasks have finished in about 1-2 hours on my Ryzen 9 3900X, but I have several that have been running for 12+ hours already with no solid estimate of when they might be completed. So it could be very easy to get stuck on some of these long tasks and have the rest of the cached tasks expire before their deadline.
ID: 7476 · Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 9 Apr 08
Posts: 841
Credit: 1,888,960
RAC: 0
Message 7479 - Posted: 10 Jan 2024, 9:46:22 UTC - in response to Message 7473.  

I'll try to contact someone by Twitter/X o by mail....

Thanks a lot.


Obviously, no answer....
ID: 7479 · Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 9 Apr 08
Posts: 841
Credit: 1,888,960
RAC: 0
Message 7487 - Posted: 23 Jan 2024, 17:07:39 UTC - in response to Message 7475.  

Meantime, if the project update the wrapper is not a bad idea: now they are using 26203 (3 years old) and the newest is 26207 (December 2023)
Since 26204 there is the multi-attach functionality


26207 is in beta
The latest stable is 26206.
It's simple enough to implement....
ID: 7487 · Report as offensive    Reply Quote

Message boards : Number crunching : Request: Increase of deadline for Virtualbox-using rosetta python app



©2024 University of Washington
http://www.bakerlab.org