The decision to change our methodology came with much consideration. PAY (Percentage Accuracy Yield) was accepted by the fantasy community and validated by numerous analysts. With that said, we decided to transition away from PAY for several reasons:
1) Our new methodology offers increased transparency. We’ve always been transparent as far as our methodology is concerned. Our new system lends itself very well to offering reports and analysis that show exactly why each expert rated the way they did. The experts have access to these reports and we plan to offer them on the site soon as well.
2) The new system is more flexible. Simply put, we’ll be able to do more with the new Accuracy Gap metric. This includes rating experts for additional scoring settings (PPR, Half PPR) as well as determining who the most accurate experts are for individual players. The additional insight should benefit fantasy owners and experts alike.
3) The PAY %s were often misinterpreted. The complexity of the PAY calculation made the output difficult to interpret. We frequently found that the %s were misconstrued as the % of time an expert made a correct pick. In actuality, the PAY values were more nuanced than that. We believe our new system represents an equally fair grading system and will be easier to interpret the results in meaningful ways.
The bottom line is our aim is to create an accuracy system that levels the playing field in evaluating experts, is fully transparent, offers results that are actionable for both fantasy experts and fantasy owners, and (ideally) does not require a PhD to interpret.