Bayesian modeling has proven its usefulness for poll aggregation and election forecasting – for instance through FiveThirtyEight. However, in Norway, media coverage of public opinion trends still tends to focus on a single poll at a time, or a simple average of polls at best. I was convinced it would be possible to do better, and after about three times as many long days and nights as I thought it would take, the result is finally live – both in Norwegian and English – at Estimite.com.
I have landed on a fully automated production process, where a script checks for new data, refits the model if necessary, and then rebuilds and publishes the website. This is all done using R, Stan, and a few shell scripts.
Here are some key lessons from this project:
Stan is great for this, because it lets us define generated quantities, and get all kinds of relevant outputs from a single, joint simulation.
You should not underestimate how many technical challenges such projects entail and how much time it takes to fix them – especially if you care about details.
[Bonus lesson] If you do work that the media find interesting, and you generally pick up the phone when journalists call, your research agenda may get completely derailed – but, hey, the problem is mostly self-inflicted.