Task 5465348

Name	RF_SAVE_ALL_OUT_NOJRAN_IGNORE_THE_REST_validation_env_g_pred_371_16903_2_1
Workunit	4851219
Created	15 Jun 2024, 15:28:31 UTC
Sent	15 Jun 2024, 15:29:54 UTC
Report deadline	16 Jun 2024, 15:29:54 UTC
Received	15 Jun 2024, 17:08:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	12 (0x0000000C) Unknown error code
Computer ID	42617
Run time	38 min 55 sec
CPU time
Validate state	Invalid
Credit	0.00
Device peak FLOPS	4.08 GFLOPS
Application version	Generalized biomolecular modeling and design with RoseTTAFold All-Atom v0.02 (nvidia_alpha) windows_x86_64
Peak working set size	1,905.34 MB
Peak swap size	6,198.90 MB
Peak disk usage	2.11 MB

Stderr output

<core_client_version>8.0.0</core_client_version>
<![CDATA[
<message>
 - exit code 12 (0xc)</message>
<stderr_txt>
Traceback (most recent call last):
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 708, in <module>
    pred.predict(out_name+f'_{n}', 
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\predict.py", line 551, in predict
    logit_s, logit_aa_s, logit_pae, logit_pde, p_bind, pred_crds, alpha, pred_allatom, pred_lddt_binned,                msa_prev, pair_prev, state_prev = self.model(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\RoseTTAFoldModel.py", line 358, in forward
    msa, pair, xyz, alpha_s, xyz_allatom, state, symmsub = self.simulator(
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 1106, in forward
    msa, pair, xyz, state, alpha, symmsub = self.main_block[i_m](msa, pair,
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 927, in forward
    pair = self.pair2pair(pair, rbf_feat, state, crop)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Track_module.py", line 368, in forward
    pair = pair + self.drop_col(self.col_attn(pair, rbf_feat)) 
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\cv2\rf2aa\Attention_module.py", line 465, in forward
    attn = einsum('bnihk,bnjhk->bijh', query, key) 
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\opt_einsum\contract.py", line 507, in contract
    return _core_contract(operands, contraction_list, backend=backend, **einsum_kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\opt_einsum\contract.py", line 591, in _core_contract
    new_view = _einsum(einsum_str, *tmp_operands, backend=backend, **einsum_kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\opt_einsum\sharing.py", line 151, in cached_einsum
    return einsum(*args, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\opt_einsum\contract.py", line 353, in _einsum
    return fn(einsum_str, *operands, **kwargs)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\opt_einsum\backends\torch.py", line 45, in einsum
    return torch.einsum(equation, operands)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\functional.py", line 297, in einsum
    return einsum(equation, *_operands)
  File "C:\ProgramData\BOINC\projects\ralph.bakerlab.org\ev0\lib\site-packages\torch\functional.py", line 299, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:79] data. DefaultCPUAllocator: not enough memory: you tried to allocate 51518208 bytes.

</stderr_txt>
]]>