Out-of-sample
Data held back from development and used once, to validate a rule.
In plain English
It only works if you genuinely do not look. Peeking and re-tuning converts it into development data.
Example
Developing on 70% of a history and testing once on the remaining 30%.