Task 5457735

Name	RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_f_pred_30_16902_3_1
Workunit	4846811
Created	13 Jun 2024, 2:06:24 UTC
Sent	13 Jun 2024, 2:37:37 UTC
Report deadline	14 Jun 2024, 2:37:37 UTC
Received	13 Jun 2024, 3:04:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	12 (0x0000000C) Unknown error code
Computer ID	48779
Run time	12 min 48 sec
CPU time
Validate state	Invalid
Credit	0.00
Device peak FLOPS	3.34 GFLOPS
Application version	Generalized biomolecular modeling and design with RoseTTAFold All-Atom v0.02 (nvidia_alpha) windows_x86_64
Peak working set size	4,027.79 MB
Peak swap size	8,687.55 MB
Peak disk usage	2.09 MB

Stderr output

<core_client_version>7.24.1</core_client_version> <![CDATA[ <message> The access code is invalid. (0xc) - exit code 12 (0xc)</message> <stderr_txt> Traceback (most recent call last): File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 708, in <module> pred.predict(out_name+f'_{n}', File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 551, in predict logit_s, logit_aa_s, logit_pae, logit_pde, p_bind, pred_crds, alpha, pred_allatom, pred_lddt_binned, msa_prev, pair_prev, state_prev = self.model( File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\RoseTTAFoldModel.py", line 358, in forward msa, pair, xyz, alpha_s, xyz_allatom, state, symmsub = self.simulator( File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 1082, in forward dchiraldxyz, = calc_chiral_grads(xyz.detach(),chirals) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1364, in calc_chiral_grads if l.item() == 0.0: RuntimeError: CUDA error: the launch timed out and was terminated CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. </stderr_txt> ]]>