Task 5462343

Name	RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_g_pred_226_16903_6_0
Workunit	4850362
Created	14 Jun 2024, 4:27:29 UTC
Sent	14 Jun 2024, 9:00:34 UTC
Report deadline	15 Jun 2024, 9:00:34 UTC
Received	14 Jun 2024, 11:42:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	12 (0x0000000C) Unknown error code
Computer ID	49160
Run time	2 hours 39 min 40 sec
CPU time	17 sec
Validate state	Invalid
Credit	0.00
Device peak FLOPS	5.25 GFLOPS
Application version	Generalized biomolecular modeling and design with RoseTTAFold All-Atom v0.02 (nvidia_alpha) windows_x86_64
Peak working set size	3,931.87 MB
Peak swap size	10,197.98 MB
Peak disk usage	2.11 MB

Stderr output

<core_client_version>8.0.2</core_client_version> <![CDATA[ <message> The access code is invalid. (0xc) - exit code 12 (0xc)</message> <stderr_txt> Traceback (most recent call last): File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 708, in <module> pred.predict(out_name+f'_{n}', File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 551, in predict logit_s, logit_aa_s, logit_pae, logit_pde, p_bind, pred_crds, alpha, pred_allatom, pred_lddt_binned, msa_prev, pair_prev, state_prev = self.model( File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\RoseTTAFoldModel.py", line 346, in forward msa_recycle, pair_recycle, state_recycle = self.recycle(msa_prev, pair_prev, xyz, state_prev, sctors, mask_recycle) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Embeddings.py", line 357, in forward dist = rbf(torch.cdist(Ca_or_P, Ca_or_P)) File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\util_module.py", line 88, in rbf D_mu = torch.linspace(D_min, D_max, D_count).to(D.device) RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. </stderr_txt> ]]>