Table 3 Optimal hyperparameters used for training the model
Input dimension | Number of heads | Dropout rate | Hidden dimension | Number of Layers | Optimizer | Learning rate |
|---|---|---|---|---|---|---|
128 | 8 | 0.1 | 512 | 6 | AdamW | 1e-5 |
Input dimension | Number of heads | Dropout rate | Hidden dimension | Number of Layers | Optimizer | Learning rate |
|---|---|---|---|---|---|---|
128 | 8 | 0.1 | 512 | 6 | AdamW | 1e-5 |