Report: Football Predictions – Twitter Against the Odds
The Business of Football Prediction
Football predictions, they are as much a part of the culture of watching football as complaining at a referee or burning your mouth on a half-time pie. From the handsomely paid ex-pro pundits reclining casually in a studio sofa to the screaming fan spilling their pint in any local pub, to the avalanche of betting firms jostling for ad space during the game, everyone’s at it; predicting results, predicting transfers, predicting (demanding) managerial changes, predicting the outright winners, and of course the outright losers – the teams engaged in the yearly relegation battle.
The football predictions business has been turned into exactly that; an extremely lucrative and increasingly cut-throat business. Since the change in UK Gambling law in 2007 that enabled betting firms to advertise to the public, it’s only taken a few short years for the prediction business to become more and more part-and-parcel of the average fan’s enjoyment of football; literally putting their money on the line to reward their own perceived knowledge of the game. There’s a growing multi-million pound industry based around this simple premise.
Fan Predictions on Twitter
But of course football fans are notoriously fickle, or so we’re told, constantly swayed week-on-week from conflicting opinion every seven days based on results. Or contrary to that idea, is there a more considered and informed trend of opinion amongst fans based on trends, performance history, and myriad other nuances and infinite variables that dictate the final results?
To find out we looked at Tweets from UK football fans from March to the end of the Premier League season in May that predicted or offered an opinion on which three teams would get relegated down to the Championship. Relegation was selected for study as opposed to say, the winner of the league, or race for Top Four due to the multitude of teams involved in potential relegation, and the fact that it was an on-going unresolved story right up until the penultimate week of the season.
Eventually the week-on-week drama played out to leave QPR and Reading relegated with three weeks to go, and Wigan joining them after an ultimately doomed post-Easter annual revival the week before the season’s end. For the three months from March to May up to nine teams slipped and slided dangerously, flirting with the drop, with no team ever really making a considered and extended push for safety, this meant that the final three to get relegated was wildly unpredictable, changing week-on-week as teams won unlikely victories and suffered surprise defeats.
We wanted to see how opinions changed as each week’s results came in and how the look of the foot of the league table morphed each weekend. How do different teams produce differing opinions? Were some teams destined to go down? Or were football fans caught by surprise?
The results can be viewed in the chart below. The bars show total number of tweets expressing whether each team would survive (positive number) or be relegated (a negative number), and by selecting each week’s results we can see how these proportions changed as the results came in.
So given that we now know QPR, Reading and Wigan were all relegated, was this signposted by the UK’s football fans?
When looking at the resulting data set above, an interesting point to note is that across the whole range of data, there was never a set of results that led to the majority opinion predicting the three correctly relegated teams.
Out of nine rounds of fixtures (until QPR & Reading were mathematically doomed on 28th April) only three times did each of the relegated teams receive a negative (to get relegated) score, and each time, there was always at least one team that public consensus talked more about getting relegated.
The main culprit for this trend was Wigan Athletic.
Wigan Athletic – The Optimists’ Choice
Every football fan in the UK knows that ‘plucky’ Wigan are renowned for their yearly ‘great escape’, i.e. struggling for nine months of the season before turning into world-beaters for the home stretch. Of course this year their luck finally ran out, but interestingly, fan belief in their Houdini-esque abilities didn’t. From looking at the data, optimism abounds around Wigan Athletic. Despite results not going their way, fans still clung to the idea that they would eventually kick into gear and get out of trouble, It’s interesting that historical performance in this case helped to blind fans to the reality that this was a below-average team, a conclusion explicitly apparent in the cold black & white of their results. They were scraping the foot of the league table for good reason. Yet this clear statistical evidence lost out to the more dubious ‘evidence’ of their survival in season’s past.
Look at Wigan’s defeat to Man City on 17 April, the loss in no way affected the resolve of fans in predicting Wigan’s survival. They continued to pick up points against the lesser teams, and this belief in them grew. Fans finally accepted Wigan’s fate when they lost a crucial game at home to Swansea on 7th May. But what’s also interesting is that ‘staying up’ predictions outweigh the negative even when they eventually got mathematically relegated against Arsenal on 14th May, such was the fan belief that they could pull off this most unlikely of victories, (and the fact that an evening kick off ensured a full day’s worth of positive tweets willing-on their survival). This is also a case of ‘wishful thinking’, which is a factor in all predictions to a certain extent. Neutral fans wanted an exciting final day relegation play off between Wigan and Aston Villa, but Wigan needed to beat Arsenal for this to occur. This would help account for the overly optimistic reaction of opinions in the build up the game. Wishful thinking can easily influence the predictions of fans, but less so the bookies.
Fans versus the Firms
An assumption one could justifiably make is that fans are a traditionally superstitious bunch, the bookies’ less so. Fans let their bias and pessimism/optimism that’s been built up over a hardy, football-following career of bitter disappointments/euphoric triumphs creep in and cloud up their judgement, whereas the betting firms are the rational, clinical and ruthlessly objective contrast. The interesting point is that the betting firms and the odds they produce are somewhat at the behest of the capricious wild-eyed fans; the more optimistic fans that back Wigan to survive, the shorter the odds the firms have to produce to compensate for their possible loss of value. Unless of course, when the fans’ predictions are so erratic, they fly in the face of the evidence.
The charts below show how fans and betting firms saw the probability of relegation each week/fixture. For the betting firms, the positions are based on the odds for each team, and the percentage chance of relegation they added up to. The Fan’s predictions are based again on total volume of tweets for each team predicting that they would survive, minus the total volume of tweets predicting they would be relegated, these were then ranked from positive to negative.
We all know Twitter is the place for snap judgements, knee-jerk reactions and sudden u-turns in opinion, so it’s not really a surprise to see the dramatic difference in the two charts above. It is immediately apparent from comparing the charts just how wildly fan opinion changes week-on-week, result-after-result, as each team’s roller-coaster of fortunes influenced a fairly fickle hive-mind of fan prediction. Compare this to the much steadier, more considered prediction ranking week-on-week from the betting odds.
Of course we also have to factor in how a fan can give a relegation prediction on Twitter without any consequence, the bookies have to be much more sensible and cautious in their predictions. What’s interesting is that in theory, both sets of predictions are based on the same factual data set – results. But of course fans will always let their own natural bias towards and against certain teams to sway their predictions, not to mention the element of wish fulfilment detailed above.
Forgetting for a moment how predictions changed or didn’t change week-on-week, who was more often right in predicting the three teams to go down? Again the betting firms show a much shrewder sense of fortune telling than the fans. As mentioned above, collective fan opinion on Twitter never had the three relegated teams as the majority favourites to go down, the betting firms were correct four out-of nine weeks before QPR & Reading were confirmed, and a further two out of three of the remaining weeks before Wigan’s ultimate relegation, adding up to a 50:50 success rate for the betting firms.
There were some correlations, both sets of predictors had QPR as more or less doomed from the get go for example. But despite the wildly varying reactions of the fans to results each week, they did trump the betting firms occasionally. Twitter swelled at reaction to the Sunderland sacking of manager Martin O Neil and the subsequent appointment of the controversial Paulo DiCanio. But amidst the extraneous opining on DiCanio’s character or previous political ideologies, the overal Twitter verdict was that Sunderland were going to survive relegation, and the immediate results in the aftermath reaffirmed this belief. The betting firms caught up to this opinion as results convinced them of DiCanio’s rejuvenating effects on a failing team, but they took more convincing. The fan predictions only had Sunderland in the bottom three once, the betting firms had them to be relegated three times. In this case, Twitter called it right.
The Future of Betting Firms & Social Media Fan Opinion: Selective Listening
In a way the predictions of both fans on Twitter and the betting firms are predictable in themselves – they are a reaction to the evidence available, i.e. results and performances of football matches. The collective knowledge and expertise of thousands of Twitter fans is a great model for collating majority opinion, but with something as emotive as sport, personal bias and preferences come into play, enough to muddy the water for accurate predictive analysis. Not to mention the very reason football is so popular in the first place – on any given day, anything can happen. The game keeps fans and bookies guessing alike; as a nailed-on victory changes unexpectedly to a shock defeat.
Personally I think that there is a benefit to monitoring social media for collective predictive opinion, and I certainly believe there’s a future for betting firms to gain a competitive edge by monitoring the wax and wane of opinions on Twitter, but it’s more likely to be found in selective listening.
As we’ve already seen, the mass of opinions of football fans is unreliable and ultimately flawed because fans by their very nature are subjective. However there is an ever-growing new breed of football analysts and experts who utilise the abundance of statistical data around every element of the game to draw shrewd and erudite conclusions and analysis. Bloggers like Michael Cox (Zonalmarking.net) are approaching the game from a position of highly informed opinion, with an objective and analytical mind.
This is where I believe the real value is to be found in gaining a competitive predictive edge on the competition; by pooling the opinions of a selective expert group of bloggers & Twitter accounts, finding the consensus on the outcomes of games and the winners and losers of tournaments. This focussing on the experts who have studied the field, may add up to a much better indicator of the overall outcome, or at the very least, give fans, or betting firms a better grasp on the probabilities. The best thing is, these experts have often made a name for themselves on Twitter and as such, we all have full access to their thoughts, insights and predictions. Taken in isolation, it’s just one experts opinion, taken with several hundred other fellow experts, you start to narrow down the probabilities.
Taking this a step further off the actual football field, the collective insider info from journalists and game insiders on new player transfers, managerial changes, etc. mean that a focussed and specific group of influential and informed Twitter accounts can futher refine the probabilities and likely outcomes of this particular element of football betting. Rumours can and will spread like wildfire amongst the uninformed masses of Twitter, but usually, (not always) by focussing on the right voices, we are able to sort the fact from the fiction, this can add a solid string to the bow of the betting firms.
Football prediction as a business demands a wealth of knowledge. By reigning in the thousands of tweets flying around offering their tuppence on the winner of the Champions League final, or the next likely manager of Man City, or Premier League relegation in 2014, sorting the informed and knowledgable voice from the doom-and-gloom of the long suffering fan, and compiling the opinions into a sharper focused probability of success or failure, could be where Twitter comes into its own as a predictive indicator for big business gambling in the future.