Task 5457838

Name	RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_f_pred_244_16902_6_0
Workunit	4848076
Created	13 Jun 2024, 2:21:15 UTC
Sent	13 Jun 2024, 6:13:33 UTC
Report deadline	14 Jun 2024, 6:13:33 UTC
Received	13 Jun 2024, 7:11:51 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	48783
Run time	41 min 50 sec
CPU time	1 min 16 sec
Validate state	Invalid
Credit	0.00
Device peak FLOPS	5.29 GFLOPS
Application version	Generalized biomolecular modeling and design with RoseTTAFold All-Atom v0.02 (nvidia_alpha) windows_x86_64
Peak working set size	3,977.43 MB
Peak swap size	8,341.92 MB
Peak disk usage	2.33 MB

Stderr output

<core_client_version>7.24.1</core_client_version> <![CDATA[ <stderr_txt> Traceback (most recent call last): File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 708, in <module> pred.predict(out_name+f'_{n}', File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 551, in predict logit_s, logit_aa_s, logit_pae, logit_pde, p_bind, pred_crds, alpha, pred_allatom, pred_lddt_binned, msa_prev, pair_prev, state_prev = self.model( File "D:\BOINC\DATA\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\RoseTTAFoldModel.py", line 358, in forward msa, pair, xyz, alpha_s, xyz_allatom, state, symmsub = self.simulator( File "D:\BOINC\DATA\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 1135, in forward dljdxyz, dljdalpha = calc_lj_grads( File "D:\BOINC\DATA\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1316, in calc_lj_grads Elj = calc_lj( File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1143, in calc_lj ljval = lj( File "D:\BOINC\DATA\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1070, in forward ai,aj,seqi,seqj,ridx = ai[mask],aj[mask],seqi[mask],seqj[mask],ridx[mask] RuntimeError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 2.00 GiB total capacity; 1.56 GiB already allocated; 0 bytes free; 1.71 GiB reserved in total by PyTorch) 09:09:39 (25712): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_f_pred_244_16902_6_0_r1110794168_0</file_name> <error_code>-240 (stat() failed)</error_code> </file_xfer_error> </message> ]]>