What is Out-of-sample (OOS)?

Probability of Backtest Overfitting (PBO). Developed by Bailey, Borwein, López de Prado and Zhu. The data is split into many pieces, which are recombined into many in-sample and out-of-sample pairs. For each pair, the procedure asks: "Did the configuration that was best in-sample end up below the median out-of-sample?" The fraction of pairs where the answer is yes is an estimate of how likely your selection process is to pick an overfit winner.