Task 5462962

Name	RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_f_pred_264_16902_2_1
Workunit	4848189
Created	14 Jun 2024, 7:04:55 UTC
Sent	14 Jun 2024, 9:36:39 UTC
Report deadline	15 Jun 2024, 9:36:39 UTC
Received	26 Jun 2024, 10:58:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	48121
Run time	3 hours 54 min 56 sec
CPU time	1 sec
Validate state	Invalid
Credit	0.00
Device peak FLOPS	4.41 GFLOPS
Application version	Generalized biomolecular modeling and design with RoseTTAFold All-Atom v0.02 (nvidia_alpha) windows_x86_64
Peak working set size	3,289.86 MB
Peak swap size	12,457.55 MB
Peak disk usage	3.09 MB

Stderr output

<core_client_version>7.16.20</core_client_version>
<![CDATA[
<stderr_txt>
Traceback (most recent call last):
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 708, in <module>
    pred.predict(out_name+f'_{n}', 
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 551, in predict
    logit_s, logit_aa_s, logit_pae, logit_pde, p_bind, pred_crds, alpha, pred_allatom, pred_lddt_binned,                msa_prev, pair_prev, state_prev = self.model(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\RoseTTAFoldModel.py", line 358, in forward
    msa, pair, xyz, alpha_s, xyz_allatom, state, symmsub = self.simulator(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 1135, in forward
    dljdxyz, dljdalpha = calc_lj_grads(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1316, in calc_lj_grads
    Elj = calc_lj(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1143, in calc_lj
    ljval = lj(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\loss.py", line 1075, in forward
    use_hb_dis = (
RuntimeError: CUDA error: unspecified launch failure
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
20:56:55 (11420): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_f_pred_264_16902_2_1_r1848905184_0</file_name>
  <error_code>-240 (stat() failed)</error_code>
</file_xfer_error>
</message>
]]>