Implementation Details (A.2.1)
SFNs are built upon transformer architectures, specifically the Multi-Layered Convolutional Framework (MCF). The MCF network consists of six transformer blocks, each comprising one cross-attention layer and two self-attention layers. Each self-attention layer contains four attention heads, allowing the model to capture complex relationships between atoms in a molecule. The Adam optimizer is employed during training with an EMA decay rate of 0.9999, ensuring stable learning dynamics.
Context and Query Sets (A.2.1)
In SFNs, the context and query sets are created by concatenating positional embeddings with the atomic positions in a molecule. The positional embedding represents the atom’s position within the molecule, which is crucial for capturing spatial information. During training and inference, the number of context and query sets is set equal to the number of atoms in the molecule.
Hyperparameters (A.2.1)
The choice of hyperparameters can significantly impact the performance of SFNs. In this study, several hyperparameters are employed:
* Learning rate: 1e-4
* EMA decay rate: 0.9999
- Number of context and query sets: equal to the number of atoms in the molecule
Conclusion (Summary)
Score Field Networks offer a promising approach to predicting molecular properties by incorporating time-step information into the score computation. By leveraging transformer architectures and positional embeddings, SFNs can capture complex relationships between atoms in a molecule. The choice of hyperparameters is crucial for optimizing model performance, and several key hyperparameters are employed in this study. Understanding the implementation details and hyperparameters used in SFNs can aid researchers in developing and applying these networks for various applications in chemistry and beyond.
