Multinational Energy Company Uses Crowdsourcing to Predict Impacting Factors of Oil & Gas Prices

By Cultivate Labs on April 18, 2017

Our clients come to Cultivate Labs with diverse needs and goals. We took a minute to sit down with a Graduate Commercial Analyst at a mutinational energy company who uses Cultivate Forecasts, to find out why they decided to incorporate internal crowdsourcing within their business, and how they have been so successful at engaging employee participation. 

(Please note: We have withheld the company and employee names to protect the innocent.)

Cultivate Labs: What prompted you to try internal crowdsourced forecasting with your company?

Secret Client: Our Business Climate and Energy Outlook team are responsible for making predictions about oil and gas prices, economic indicators, and geopolitical events. The team recognizes the importance of the ‘wisdom of the crowd’ in forecasting. In addition to improving our internal predictions, we also aim to encourage employees to learn more about the external environment in which our company operates.

After researching and reading literature about crowdsourced forecasting, Cultivate Labs seemed like a perfect solution for an internal crowdsourcing platform, and we are currently in the midst of a year-long forecasting challenge using Cultivate’s software.

CL: What was management’s initial reaction to using internal crowdsourcing as a new approach to forecasting? Has it changed one way or another since you began the exercise?

Secret Client: Management has been supportive of our team instituting a prediction market challenge right from the beginning, and they continue to support it. We even published an article in the company newsletter, so employees and leadership not participating could learn more about the challenge.

CL: How do you hope to use the insights you are generating, if not now, then in the future?

Secret Client: Our Business Climate and Energy Outlook team will use the insights we uncover as an additional point of reference when presenting forecasts or research papers to our Executive Committee. They will allow us to review possible sentiment analysis effects when forecasting energy prices.

We hope to be able to chart the information and use it in presentations in the future as an example of employee engagement with external business factors.

CL: You have very high participation rates. What do you think the key lessons learned have been to getting people to participate on a regular basis?

Secret Client: My number one tip is to bringing out participants’ competitive sides.

We are awarding prizes based on both a participant’s net worth and accuracy score. Active participation is encouraged in order to generate a higher net worth.

We are also awarding a trophy to the best performing department. The forecasting challenge is open to all employees, so we asked users to make their profile name something related to their department. When viewing the leaderboard, participants can see which department have members who are performing well.

Participation rates have been highest for questions where more than two outcomes are possible. There is constantly new information being circulated in the news, and this gives people a reason to constantly update their forecast, for example in regards to predicting oil prices. Because there are many different outcomes for forecasting prices, users often look for arbitrage opportunities – hence high participation.

Our questions are topical (e.g. political events), and most participants who signed up are interested in geopolitics. They all feel they have enough insights to answer the questions from reading the daily news. I feel the key for other admins using Cultivate Forecasts is to create questions that a wide audience may have some knowledge about and be interested in. The software works better with more participants to increase liquidity.

Finally, I distribute monthly newsletters to participants with suggested related reading and a current leaderboard categorized by departments. I also remind users when new questions are being released, which encourages people to place or update forecasts.

CL: What has the team’s reaction been like to the forecasting challenge?

Secret Client: The participants are really enjoying the challenge, and the majority are still actively participating. They have found the platform very easy to use, and I have had minimal help questions asked. We have a lot of healthy competition with an average of 124 forecasts per question (note we have 100 users total– users are able to make multiple predictions per question).

We will have a prize at the end of the challenge for the participant who comes in the top 10 and also has the best overall rationale. Because of this rationale stipulation, we have healthy discussions and comments in the rationale section, and this has taught some users how to make better forecasts and what drivers to analyze.

The challenge includes questions that cover a range of topics, so people are able to contribute to questions where they feel they may have better knowledge or interest.

Participants have been overwhelmingly positive about the experience, and one said “I’ve always had a strong interest in geopolitics. This challenge further spurs my interest and gives me a fun and competitive outlet to test my forecasting abilities.”

CL: Similarly, has the relationship or dynamic of the group between management and staff changed noticeably since you started this exercise?

Secret Client: We have not yet noticed a change in the relationship between management and staff as a result of the challenge, but we wouldn’t be surprised if more people start frequent dialogues after we publish some of the results in our quarterly Executive Communication reports.

CL: Have any of the results of the questions you’ve asked gone against the “conventional wisdom” that already existed within the organization about a particular topic?

Secret Client: Not yet, our questions typically cover a longer time period so we’ve only had a couple resolve so far. We’d love to see if any results arise in the future that challenge how we’ve always approached thinking about these events and forecasts and prompt a new way of thinking.

CL: How did you previously make forecasts? Were those forecasts ever measured for accuracy? What is your reaction to how accurate/inaccurate your “crowd” has been using crowdsourced forecasts?

Secret Client: Our team has our own data driven models to forecast energy prices. For geopolitical events we would look to other crowdsourcing platforms such as Nate Silver’s FiveThirtyEight. The team would routinely benchmark the models and review the model diagnostics.

Unfortunately, we have only had two questions resolve thus far, so not a very good sample size for us to know how accurate (or inaccurate) the crowd’s forecasts are yet. I will say however, the crowd accurately predicted the answer at least 20 days preceding the end date in both questions, so that’s a promising indicator of their accuracy.

CL: Now that you’ve had some experience with crowdsourced forecasting in your group, where else do you think crowdsourced forecasting could be used in the company?

Secret Client: We have a maximum of 100 participants in our challenge in a company of ~3,500 people, so there is definitely scope for other business units to use crowdsourcing internally. Some examples may include how to improve the company’s data science projects or the expected success rate of new drilling programs.

CL: Finally, do you have any advice or lessons learned for other companies or groups who are looking to source better insights from their teams through crowdsourced forecasting and how to be successful?

Secret Client: One piece of advice I have is to run a trial question or a TinyCast at the beginning so that participants can try using the platform before jumping into a challenge. That being said, I only had four users register for the training sessions I offered because the software is very intuitive to use. Most users were happy to learn by doing instead.

I would also advise having a mixture of questions with yes/no answers as well as questions with more than two possible answers. The questions with more than two answers generate more discussion and show viewpoints that the forecasting team may not have previously considered. These alternate viewpoints can often be the most useful output of crowdsourcing.

You might be interested in: Have your team predict your KPI’s in real-time so you can actually meet your KPI’s.

prediction markets change management disruptive leadership crowdsourced forecasting client spotlights