While it seems like my forecasts are making markets less accurate, this turns out not to be the case.  This is because my "incorrect" forecasts tend to make markets slightly less accurate, whereas my "correct" forecasts make markets much more accurate.  To explore this, I looked at how my forecasts affected markets' Brier Scores, our usual metric for forecast accuracy.

Brier scores evaluate markets by aggregating all the forecasts throughout their history, so I compared the Brier component of each market before and after I forecasted and calculated their difference.  The following chart shows the distribution of my forecasts:

While the majority of forecasts result in accuracy declines, the declines are relatively small, mostly resulting in less than a 0.05 decline in Brier Score.  In contrast, there are a substantial number of very large improvements, which more than offset the declines.  273 forecasts improve the Brier Score by at least 0.1, compared to only 76 that decrease it comparably.  In aggregate, my forecasts are making markets more accurate, even though the majority of my forecasts are in the wrong direction.

In upcoming blog posts, I'll dig deeper into the data to better understand which of my forecasts are most valuable, where I might be making mistakes, and how I can improve.