2024 End of Year Retrospective

By Adam Siegel on December 19, 2024

Before we close shop and take a few days off, I wanted to take a moment to reflect on the challenges, milestones, and relevant events that defined the year for our company + a few extra musings if you will so indulge me. So let’s take a step back and reflect on what made this year what it was.

Our Impact

Our business model provides a hosted or on-premise forecasting platform to enable organizations to run their own crowdsourced forecasting efforts internally and externally. Alongside that technology we offer consulting services to make sure the use of our platform is optimized in whatever scenario it is being used, both technologically and most importantly, as part of decision making processes. I want to highlight 6 of our projects that I felt made a particular impact this year (and that I can publicly discuss!)

RAND Forecasting Initiative (RFI)

Having its lineage in CSET’s Foretell, the RAND Corporation transitioned the University of Maryland’s ARLIS INFER program to create a new offering for government stakeholders in the U.S. and abroad. After a competitive bidding process, Cultivate was chosen to support RAND in their efforts. The ultimate impact of that work will be over the course of years, but RAND brings a unique ability to bring RFI forecasts to a massive portfolio of their own projects which will flow directly to their government clients. Already this year several projects have launched questions and leveraged results in their deliverables, and plans are to scale this massively in 2025. Here is a year-end review on the RFI blog if you’d like to learn more. And you can make an impact too by signing up to forecast.

Fox News

Launched in 2022, Cultivate runs a forecasting effort for Fox News called “America Predicts.” Amassing tens of thousands of forecasts each week from now well over a million people, Fox News editors place an embeddable Cultivate widget to ask a forecast question relevant to the article. Readers use a custom “needle” interface to set a probability, and can also provide a rationale for their forecast. I don’t know the traffic at Kalshi, Polymarket, and other large public prediction sites, but I think it’s safe to say this is one of the largest efforts to expose the task of probabilistic forecasting to a massive number of people right now. A roundup of all currently running questions across Fox News articles can be seen here: foxnews.com/americapredicts

The Alliance for Decision Education’s Forecasting the Future Forecasting Tournaments

From their website, the Alliance for Decision Education (ADE) is “catalyzing a transformative movement to empower students with the skills and dispositions for making better decisions. Decision education prepares students to flourish in our ever-changing world. It’s about teaching students how to think, not what to think.” This year, Cultivate supported ADE in running two forecasting tournaments with hundreds of participants from High Schools around the U.S. For each tournament, students are rewarded cash prizes for their accuracy in forecasting, but more importantly, all participants are being taught critical thinking skills through the act of forecasting. You can read the wrap-up of the first tournament this year in this ADE blog post.

Canadian Forest Service’s “Firesight”

As a hiker who has experienced the beauty of Canada’s vast forest ranges, I was saddened to see so many uncontrolled forest fires in 2024. The Canadian Forest Service approached Cultivate to see if we could help their community of firefighters, forest managers, and forest fire researchers create more accurate long-term forecasts for fires. Better forecasts would mean better planning and resource utilization. From our blog post earlier this year about the project:

“The wildfire management community has always been in the business of predicting and preventing fires – but the situational and dynamic nature of wildfires poses a variety of challenges for forecasting both in the short term and the longer term. Seasonal forecasting is thought by many in the community to not be possible with any degree of skill, yet many in the community engage in watercooler conversations prior to the wildfire season about what they expect to come in the year…With Firesight, we are able to tap into a broader pool of knowledge and perspectives, which is especially valuable given the regionally varied nature of wildfires across Canada. We ultimately hope to improve our wildfire predictive capabilities.”

Bertelsmann RANGE

The North American contingent of the Bertelsmann Stiftung in partnership with their German colleagues in Berlin lead RANGE (Rethinking Assumptions in a New Geostrategic Environment) a “crowdsourced forecasting program focused on geopolitics and the transatlantic relationship.” RANGE, completing its 3rd year, ran multiple tournaments across U.S. and European college campuses, and brought the concept of crowdsourced forecasting to forums in Madrid, Berlin, and Athens. RANGE forecasts and associated methodology continue to make their way into foreign offices (U.S. State Department equivalents) across Europe.

The UK Government’s “Cosmic Bazaar”

With a direct line of communication to analysts working with senior decision makers throughout the national security community, the thousands of civil servants who have made forecasts in Cosmic Bazaar have brought a critical outside perspective to the decision making process. You can read more about Cosmic Bazaar in this oldie but goodie Economist Article by Shashank Joshi: “How Spooks Are Turning to Superforecasting in the Cosmic Bazaar.” As an aside, one rarely hears about good experiences working with bureaucracy within governments, but our experience has been excellent. We continue to be impressed with the responsiveness of the various digital teams we interact with to provide the hosting and technical support to the platform. And by the way, we were approved to be a UK Government Cloud Provider by the Commercial Crown Service so it’s really easy to access our services if you’re from HMG!

What’s Coming in 2025

I’m pleased about our pipeline of potential clients in 2025 and what I already know is coming on the forecasting front, but I’m particularly excited about what I feel may turn into a big opportunity for us in the coming months and years.

In our last newsletter, we gave a hint at a new product we’ll be launching shortly, ARC (Analyst Research Companion.) Currently intended for use by geopolitical analysts, macro analysts at financial institutions, and journalists, ARC is a AI/human hybrid application designed to instantly map drafts of possible scenarios about future events, define market and geopolitical drivers and indicators, monitor the global information landscape, and neutralize analytical bias in your writing (and discover it in others’.) We have been hard at work on ARC for the past 9 months and have already deployed it to a couple of our partners and a government intelligence service as a proof of concept. Expect to hear more about this in January, and if you just can’t wait until the official launch, you can still get a sneak peak at arcanalysis.ai.

Another offering we will launch soon is a new forecaster training curriculum we’re calling the Cultivate Forecaster Learning Lab. It builds upon work we did on INFER with Pytho, pioneering research from the Good Judgment Project and Phil Tetlock, and our own experience teaching basic forecasting concepts to employees at organizations we work with. The Learning Lab offers a free, online, comprehensive training curriculum designed to teach skills to improve your forecasting abilities. Based on what we have evaluated as best in class open source resources, the training curriculum will roll out on a Cultivate training forecasting site we will launch early next year. You will be able to take courses and knowledge quizzes based on those courses, track your progress, and practice your craft on a few sample questions. As you progress you will be rewarded badges you can flaunt on your LinkedIn profile as testimony to your new skills.

Other Reflections

The Seemingly Never Ending Prediction Market Hype Cycle

For better and worse, I have been working in this space since 2006 when Nate Kontny and I got funding from Y Combinator and decided to launch Inkling Markets, a company that built hosted prediction markets. In other words, I have lived through 5 U.S. Presidential election cycles and watched with glee, amusement, and now skepticism, the hype cycle prediction markets go through every 4 years. Along with the prediction markets being an alternative (valuable!) source of information, come the inevitable blog posts, newsletters, and Twitter/X posts claiming prediction markets are here to stay and they will fundamentally change the information landscape and perhaps even how we govern and manage companies. This seemed to be particularly acute this year as prediction markets both leaned in the “right direction” and contradicted some polls to give a more extreme signal of outcome.

Kalshi, Polymarket, and Manifold, 3 of the most heavily used prediction markets are slicker and more sophisticated versions of platforms available from years past. Their leadership is more savvy at marketing, social media promotion, and challenging the conventional norms through those mediums. There were lucrative story lines about rich people making outlandishly large and risky trades. There are even broader trends of young men in particular doing more sports betting and stock trading, of which prediction markets are a near cousin to.

But even with all that going for them during this election cycle, I suspect traffic at these prediction markets has all dropped significantly in the 2 months since the election and articles about prediction markets having “arrived” have largely ceased. Perhaps with an incoming U.S. administration likely to be more friendly to de-regulation of real money prediction markets, higher quality companies running these efforts, and integration with other financial platforms, the next 4 years will see a scale of uptake we’ve never seen before. But history has not been kind to these exact predictions before, so I'd still wager against it.

Forecasting Landscape

There are some very large philanthropic funders interested in forecasting who have spent tens of millions of dollars funding various research initiatives. As a self-funded, for-profit entity living and dying by being practitioners selling our wares and services, we aren’t necessarily targets for direct philanthropic funding. Nevertheless, we find ourselves at the tip of the spear working with non-profit, governmental, and commercial organizations in the application of forecasting. Here are some areas we will be focusing on that I also think the space/industry badly needs:

Increasing funding and opportunities for “Pro” forecasters. Good Judgment was the pioneer in the “Superforecaster” model. We’ve evolved it further on our RFI project with a Pro program that mobilizes these high-performing forecasters around a dedicated mission and connects them to the subject matter experts their forecasts support. The program is the one incentive and development model in non-monetary markets/forecasting efforts that has consistently worked over the myriad of others we and anyone else have tried.
Empowering decision-makers as recipients of crowd forecasts. There’s a critical need to develop comprehensive methodologies for integrating crowd forecasting into decision making processes. This includes support for leveraging forecasts under different decision-making contexts or scenarios, a “how-to” management guide, and potentially training courses. So much effort has been expended on the forecaster. Now it's time to focus on the decision maker.
AI-driven “co-pilots” for forecasters. Automated forecasters via AI will have their place, but human perspective continues to be critical, and we should be able to use AI as a companion to drive positive behavior. Stay-tuned for some of these on our forecasting platform this coming year.

An Intense Year

I’m not afraid to admit this past year has been a particularly challenging one for me, with many personal, emotional challenges along the way. And I know the same has been true for several others on our team as well. After co-running companies for almost 2 decades, I seem to have at least one fundamental reminder each year about how to try and run our company most effectively. This year was a profound, yet simple one: give space and be supportive to people going through some shit.

Nothing we do at Cultivate is as important as any of our team’s personal well-being. It’s important to be a human first, business leader second. It’s as simple as that. We lionize being “hardcore” and there are plenty of forced back to the office initiatives if you want to work in that kind of environment, but that’s not the kind of company we want to be. We work hard, but it turns out by simply being a little more human in your culture, you can engender loyalty, hard work, and camaraderie to create a company that has more profits than one of the largest social networks.

I’m so proud of our team and how we support each other, both in a work context and a personal one, and I hope to keep doing my part in creating that kind of atmosphere in the coming year.

Thank you for reading this, and as always, feel free to reach out to me with any ideas, questions, or critiques. I’d love to hear from you.

Happy Holidays and a Happy New Year,

Adam