regression dropout Transformer
regression based TensorFlow implementation for classification hyperparameter.
- Input
- 3547-dim embedding
- Encoder
- 12 x Transformer with 10 heads
- Output
- auc-roc projection
Training config
optimizer=Adagrad, lr=0.678, scheduler=plateau, warmup=1754