Price rise accuracy caution

FantasyFootballFix (FFF) prominently displays its Current Algorithm Accuracy on its player price change prediction page. I have seen some FPL managers refer to this percentage and suggest that is the proportion of price changes they are correctly predicting. That is a mistake.

This post is not a criticism of FFF; I have great respect for the great work the team behind it do in trying to reverse engineer the FPL price change mechanism to predict when players will rise and fall. Rather this is just a caution to FPL managers to be careful about their use of the algorithm accuracy statistics.

FantasyFootballFix algorithm accuracy 11 Sept 2015

One look at the Recent Price Changes data, which I think FFF uses to create the accuracy figure, shows the website doesn’t correctly predict 94.8 of price changes. In the data listed, there had been five recent risers and only two of them were at 100% or more according to FFF. So only 40 per cent of those risers were predicted by the site. If you take into consideration those players who were predicted to rise but didn’t, like Pedro, the percentage would drop further.
FFF record of risers on 11 Sept 2015
The algorithm accuracy figure appears instead to be achieved by adding up the percentages FFF had the players at when they rose and dividing it by the sample size. For example, when I looked at the figures they showed Vincent Kompany rose when FFF had him at 92.9 per cent, Joel Ward at 82.0 per cent, Marc Albrighton at 100.0 per cent, Matt Targett at 99.1 per cent and Callum Wilson at 100.0 per cent. Add those values together and divide by five and you get 94.8 per cent. (I couldn’t repeat that with the fallers – I came up with -85.9 per cent – but it might be that the larger sample size (34) allowed FFF to exclude one or two outlying samples at either extreme when calculating the fall accuracy.)

As the name “Current Algorithm Accuracy” suggests, the figure appears to tell us the average of the percentages the recent risers were on when they changed price. But, if you want to be extra careful, the recent evidence also suggests any player at 82 per cent or higher on FFF could be in the zone for a price rise.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s