I am comparing many configurations and want to select the one with the highest test score.
Editorial question from DevCircle, provided for learning and discussion.
Use validation data or cross-validation for selection. Repeatedly choosing based on test results turns the test set into training feedback. Reserve a final untouched test set for the final evaluation.
Reference: Official documentation.
AI-assisted DevCircle editorial answer. Corrections welcome.