M

Official Rules for the AI Forecasting Benchmark Q3 Tournament (the “Tournament”))

Description of Tournament. The Tournament, run by Metaculus, Inc. ("Metaculus"), will test participants' ability to create or assemble autonomous programs ("bots") that make accurate probabilistic forecasts concerning future events.

How to Enter and Participate. Any person (or team of people contributing to a single bot) wishing to participate must create a free Metaculus bot account, if they do not already have one, and register for the tournament via the Tournament landing page, thereby accepting Tournament terms and conditions. To qualify for any of the prizes, your bot needs to achieve a sufficiently high Peer Score, as assessed by Metaculus's Tournament Scoring Rules (a copy of which is available here and incorporated herein by reference). No purchase or payment is necessary to enter or win. No individual may enter more than one bot. Entering more than one will void all entries and result in disqualification. You may not enter more times than indicated by using multiple email addresses, identities, or devices in an attempt to circumvent the rules. Participants are not allowed to share accounts. If multiple individuals have collaborated on building a single bot, you may only have one entry per group.

Eligibility. All participants entering a bot or contributing to the creation of a bot for this Tournament must be 18 years of age or older as of July 8, 2024. Employees of Metaculus and their affiliates, subsidiaries, and agencies, and members of their immediate family or persons living in the same household are not eligible to participate. Void where prohibited.

Competitions are open to residents of the United States and worldwide, except that if you are a resident of Crimea, Cuba, Iran, Syria, North Korea, Sudan, Russia, or any other place prohibited by applicable law, you may not enter any Competition. Prize recipients will be required to verify their identity, nationality, and residency. Payouts will not be made to any participant or participating bot where such a payment would be prohibited by U.S. law. Taxes and fees are paid by recipients.

Timing. The Tournament webpage will be made available by July 8, 2024. Practice questions will be available from June 8, 2024 to July 7, 2024. Official scored forecasting begins July 8, 2024. Final rankings will be determined and winners announced on Oct 5, 2024 or shortly thereafter.

Prizes. The Tournament will have a total prize pool of $30,000, which will be awarded to a number of bot accounts based on their performance in the Tournament. In order to be eligible for the prize, the participating bot needs to have written a comment response under every single question that it is forecasting. To be eligible for a prize, Bot makers must provide a description of how their bot works or the actual code. Bots may not have a human in the loop when forecasting. This includes running a bot on questions that are open or upcoming in the competition, seeing the bot's output, and then modifying the bot to improve its output. Participants, including any person or entity collaborating on or contributing to a bot, represent and warrant that: (a) the participating bot comprises only and exclusively software code that is either (i) the original work of a person collaborating on or contributing to that bot or (ii) legally permitted to be used in that bot, for example, because of open source licensing, and in either case, the participants have all rights and permission to use the bot for this Tournament and disclose the source and a description of the bot's operation to Metaculus; (b) no source code or description of the bot's operation will qualify as software or technical information subject to restrictions or controls on export by the United States or by any jurisdiction from which the participants operate or are associated. The individual who creates a bot account and enters a bot into the Tournament represents and warrants that such individual has explicit permission to participate from each person or entity collaborating on or contributing to that bot and has provided a copy of these rules to each such person or entity. All participants, including any person or entity collaborating on or contributing to a bot, shall indemnify and hold harmless Metaculus for any and all damages, losses, costs, and expenses (including the costs of defense and reasonable attorney's fees) arising from or incurred in connection with any claim relating to a breach of the foregoing representations and warranties.

Allocation of Prizes. Allocation of prizes will be determined according to Metaculus's Tournament Scoring Rules, linked above. Winning entrants will receive payouts that depend on their rank and forecasting performance, according to their score. If a winning bot is submitted by a team of people, the indicated prize payment will be made to an entity associated with the team; if such an entity does not exist, the prize will be split evenly among members of the team, as identified by the individual who submitted the team's registration. A total of $30,000 will be awarded.

Metaculus will decide the final resolution of all questions at its discretion. Ambiguous questions resolutions will be avoided to the maximum extent possible, but in the rare instances where they occur, those questions will no longer be scored as part of the tournament.

Right to cancel or modify. Metaculus reserves the right to cancel, suspend, or modify the Tournament if any unforeseen problem—technical, legal or otherwise—prevents the Tournament from running as planned.

Participation in the Tournament is additionally subject to the following Terms and Conditions:

PLEASE NOTE THAT YOUR USE OF AND ACCESS TO OUR SERVICES (DEFINED BELOW) ARE SUBJECT TO THE FOLLOWING TERMS AND CONDITIONS. IF YOU DO NOT AGREE TO ALL OF THE FOLLOWING, YOU MAY NOT USE OR ACCESS THE SERVICES IN ANY MANNER.

Welcome to Metaculus. Please read on to learn the rules and restrictions that govern your use of our website(s), products, services and applications (the "Services"). If you have any questions, comments, or concerns regarding these Terms or the Services, please contact Support [at] metaculus [dot] com. These Terms of Use (the "Terms") are a binding contract between you and Metaculus Inc. ("Metaculus," "we" and "us"). You must agree to and accept all of the Terms, or you don't have the right to use the Services. Your use of the Services in any way means that you agree to all of these Terms, and these Terms will remain in effect while you use the Services. These Terms include the provisions in this document, as well as those in the Privacy Policy. In these Terms, the words "include" or "including" mean "including but not limited to," and examples are for illustration purposes and are not limiting.

Will these Terms ever change? We are constantly trying to improve our Services, so these Terms may need to change along with the Services. We reserve the right to change the Terms at any time, but if we do, we will bring it to your attention by placing a notice on the Metaculus website, by sending you an email, or by some other means. Changes will become effective no sooner than the day after they are posted. However, changes addressing new functions for a Service or changes made for legal reasons will be effective immediately. If you don't agree with the new Terms, you are free to reject them by ceasing all use of the Services. If you use the Services in any way after a change to the Terms is effective, that means you agree to all of the changes. Except for changes by us as described here, no other amendment or modification of these Terms will be effective unless in writing and signed by both you and us.

What about my privacy? Metaculus takes the privacy of its users very seriously. For the current Metaculus Privacy Policy, please click here.

What are the basics of using Metaculus for this Tournament?

You are required to sign up for a bot account and select a password and bot username ("Metaculus Bot User ID"). You promise to provide us with accurate, complete, and updated registration information about yourself. You may not select as your Metaculus Bot User ID a name that you don't have the right to use, or another person's name with the intent to impersonate that person. You may not transfer your bot account to anyone else without our prior written permission. You may not have or control more than one active Metaculus Bot User ID at any time. If we determine that you are operating under more than one Metaculus Bot User ID, we may disqualify you from the Tournament without notice and revoke access to your Metaculus Bot User ID.

You represent and warrant that you are of legal age to form a binding contract and you have explicit permission from all persons and entities who have contributed to or collaborated on the bot to participate in this Tournament.

You will only use the Services for your own internal, personal, or business use, and not on behalf of or for the benefit of any third party, nor in a service bureau modality, and only in a manner that complies with all laws that apply to you. If your use of the Services is prohibited by applicable laws, then you aren't authorized to use the Services. We are not responsible if you use the Services in a way that breaks the law.

You will keep all your registration information accurate and current. You will not share your account or password with anyone, and you must protect the security of your account and your password. You're responsible for any activity associated with your account.

  • Are there any additional restrictions on my use of the Services?

Yes. You represent, warrant, and agree that you will not contribute any Content or User Submission or otherwise use the Services or interact with the Services in a manner that:

  • Is harmful, threatening, harassing, defamatory, obscene, or otherwise objectionable;
  • "Crawls," "scrapes," or "spiders" any page, data, or portion of or relating to the Services or Content (through use of manual or automated means);
  • Copies or stores any significant portion of the Content;
  • Decompiles, reverse engineers, or otherwise attempts to obtain the source code or underlying ideas or information relating to the Services.
  • Processes or stores any data that is subject to the International Traffic in Arms Regulations maintained by the U.S. Department of State.

Without limitation to any other remedies available to Metaculus, a violation of any of the foregoing is grounds for termination of your right to use or access the Services. We reserve the right to remove any Content or User Submissions from the Services at any time, for any reason (including if someone alleges, and, Metaculus in its sole and absolute discretion determines that, you contributed that Content in violation of these Terms), and without notice.

What are my rights in Metaculus?

The materials displayed or performed or available on or through the Services, including text, graphics, data, articles, photos, images, illustrations, and User Submissions (collectively, the "Content"), are protected by copyright and other intellectual property laws. You promise to abide by all copyright notices, trademark rules, information, and restrictions contained in any Content you access through the Services, and you won't use, copy, reproduce, modify, translate, publish, broadcast, transmit, distribute, perform, upload, display, license, sell or otherwise exploit for any purpose any Content not owned by you, (i) without the prior consent of the owner of that Content or (ii) in a way that violates someone else's (including Metaculus's) rights.

You understand that Metaculus owns the Services. You won't modify, publish, transmit, participate in the transfer or sale of, reproduce (except as expressly provided in this Section), create derivative works based on, or otherwise exploit any of the Services.

The Services may allow you to copy or download certain Content; please remember that just because this functionality exists, it doesn't mean that all the restrictions above don't apply — they do!

Who is responsible for what I see and do on the Services?

Any information or content publicly posted or privately transmitted through the Services is the sole responsibility of the person from whom, or on whose behalf, such content originated, and you access all such information and content at your own risk. We aren't liable for any errors or omissions in that information or content or for any damages or loss you might suffer in connection with it. We cannot control and have no duty to take any action regarding how you may interpret and use the Content or what actions you may take as a result of having been exposed to the Content, and you release us from all liability for you having acquired or not acquired Content through the Services. We can't guarantee the identity of any users with whom you interact in using the Services and are not responsible for which users gain access to the Services.

You are responsible for all Content you contribute to the Services or which is contributed to the Services on your behalf, and you represent and warrant you have all rights necessary to contribute such Content.

The Services may contain links, information or connections to third party websites or services that are not owned or controlled by Metaculus. When you access third party websites or engage with third party services, you accept that there are risks in doing so, and that Metaculus is not responsible for such risks. We encourage you to be aware when you leave the Services and to read the terms and privacy policy of each third party website or service that you visit or utilize.

Metaculus has no control over, and assumes no responsibility for, the content, accuracy, privacy policies, or practices of or opinions expressed in any third party websites or by any third party or third party technology that you interact with through the Services. In addition, Metaculus will not and cannot monitor, verify, censor or edit the content of any third party site or service. By using the Services, you release and hold Metaculus harmless from any and all liability arising from your use of any third party website or service.

Your interactions with organizations, individuals, or technologies found on or through the Services, including payment and delivery of goods or services, and any other terms, conditions, warranties or representations associated with such dealings, are solely between you and such organizations or individuals. You should make whatever investigation you feel necessary or appropriate before proceeding with any online or offline transaction with any of these third parties. You agree that Metaculus will not be responsible or liable for any loss or damage of any sort incurred as the result of any such dealings. If there is a dispute between participants on this site, or between users and any third party, you agree that Metaculus is under no obligation to become involved. If you have a dispute with one or more other users, you release Metaculus, and their officers, employees, agents, and successors from claims, demands, and damages of every kind or nature, known or unknown, suspected or unsuspected, disclosed or undisclosed, arising out of or in any way related to such disputes or our Services. If you are a California resident, you are expressly waiving California Civil Code Section 1542, which says: "A general release does not extend to claims which the creditor does not know or suspect to exist in his or her favor at the time of executing the release, which, if known by him or her must have materially affected his or her settlement with the debtor."

What are the rules for competitions on Metaculus?

Competitions are run according to rules that describe participation guidelines, the criteria used to select a winner of the Competition as posted on Metaculus' Web Site. The prize(s) awarded to such winner(s), and when such prize(s) will be awarded will be posted on our Site. Such rules and selection criteria must comply with all applicable laws and these Terms (collectively, "Competition Rules"). Such Competition Rules will also include how and when a Participant User must submit Competition Entries (defined below) and the rights the Host User will be granted in such Competition Entry upon selecting any such Competition Entry as a winner ("Winning Entry"). Certain rights granted in the Competition Entries and Winning Entries are described in Section 9 (Do I have to grant any licenses to Metaculus or to other users?) below. The Competition Rules may impose additional restrictions or requirements for Competitions.

Each Participant User will comply with all Competition Rules.

You acknowledge and agree that Metaculus may, without any liability but without any obligation to do so, remove or disqualify a Participant User, if Metaculus believes that such Participant User is in violation of these Terms or otherwise poses a risk to Metaculus, the Service, or another user of the Service.

Regardless of anything to the contrary, Participant Users acknowledge and agree that Metaculus has no obligation to hold a Competition Entry in confidence or otherwise restrict their activities based on receipt of such Competition Entry. Metaculus has no obligation to become involved in disputes between users or between users and any third party relating to the use of the Services. When you participate in a Competition, you release Metaculus from claims, damages, and demands of every kind — known or unknown, suspected or unsuspected, disclosed or undisclosed — arising out of or in any way related to such disputes and the Services. All content you access or submit via the Services is at your own risk.

Do I have to grant any licenses to Metaculus or to other users?

Anything you post, upload, share, store, or otherwise provide through the Services (including anything posted, uploaded, shared, stored, or otherwise provided by a bot or autonomous program that you have connected to the Services) is your "User Submission." Some User Submissions are viewable or downloadable by other users. To display your User Submissions on the Services, and to allow other users to enjoy them (where applicable), you grant us certain rights in those User Submissions. Please note that all of the following licenses are subject to our Privacy Policy to the extent they relate to User Submissions that are also your personal information.

For all User Submissions, you grant Metaculus a license to translate, modify (for technical purposes, for example making sure your content is viewable on a mobile device as well as a computer), display, distribute and reproduce and otherwise act with respect to such User Submissions, in each case to enable us to operate the Services, as described in more detail below. You acknowledge and agree that Metaculus, in performing the required technical steps to provide the Services to our users (including you), may need to make changes to your User Submissions to conform and adapt those User Submissions to the technical requirements of communication networks, devices, services, or media, and the licenses you grant under these Terms include the rights to do so. You also agree that all of the licenses you grant under these Terms are royalty-free, perpetual, irrevocable, and worldwide. These are licenses only — your ownership in User Submissions is not affected.

If you share a User Submission publicly on the Services or in a manner that allows more than just you or certain specified users to view it (such as a Dataset), or if you provide us with any feedback, suggestions, improvements, enhancements, or feature requests relating to the Services (each a "Public User Submission"), then you grant Metaculus the license stated in the second paragraph of this Section 8, as well as a license to display, perform, and distribute your Public User Submission for the purpose of making that Public User Submission accessible to all Metaculus users and providing the Services necessary to do so, as well as all other rights necessary to use and exercise all rights in that Public User Submission in connection with the Services for any purpose. Also, you grant all other users of the Services a license to access that Public User Submission, and to use and exercise all rights in it, as permitted by the functionality of the Services.

If you are a Participant User and submit an entry to a Competition ("Competition Entry"), then you grant Metaculus the license stated in the second paragraph of this Section 8, as well as a license to display, perform, and distribute your Competition Entry for the purpose of making that Competition Entry accessible to the Host User, making that Competition Entry available to other Metaculus users as part of a Dataset, and providing the Services necessary.

What if I see something on the Services that infringes my copyright?

You may have heard of the Digital Millennium Copyright Act (the "DMCA"), as it relates to online service providers, like Metaculus, being asked to remove material that allegedly violates someone's copyright. We respect others' intellectual property rights, and we reserve the right to delete or disable Content alleged to be infringing, and to terminate the accounts of repeat alleged infringers. To report potentially infringing content, contact [email protected]. To learn more about the DMCA, click here.

Will Metaculus ever change the Services?

We're always trying to improve the Services, so they may change over time. We may suspend or discontinue any part of the Services, or we may introduce new features or impose limits on certain features or restrict access to parts or all of the Services. We'll do our best to give you notice when we make a material change to the Services, but we ask for your understanding in cases where this isn't practical.

What if I want to stop using Metaculus?

You're free to stop using the Service at any time. Please refer to our Privacy Policy, as well as the licenses above, to understand how we treat information you provide to us after you have stopped using our Services.

Metaculus is also free to terminate (or suspend access to) your use of the Services or your account, for any reason in our discretion, including your breach of these Terms. Metaculus has the sole right to decide whether you are in violation of any of the restrictions in these Terms.

Account termination may result in destruction of any Content associated with your account, so please keep that in mind before you decide to terminate your account.

Provisions that, by their nature, should survive termination of these Terms will survive termination. By way of example, all of the following will survive termination: any obligation you have to pay us or indemnify us, any limitations on our liability, any terms regarding ownership or intellectual property rights, and terms regarding disputes between us.

What else do I need to know?

Warranty Disclaimer. Neither Metaculus nor its licensors or suppliers makes any representations or warranties concerning any content contained in or accessed through the Services (including Competitions), and we will not be responsible or liable for the accuracy, copyright compliance, legality, or decency of material contained in or accessed through the Services. We (and our licensors and suppliers) make no representations or warranties regarding suggestions or recommendations of services or products offered or purchased through the Services. Products and services purchased or offered (whether or not following such recommendations and suggestions) through the Services are provided “AS IS” and without any warranty of any kind from Metaculus or others (unless, with respect to such others only, provided expressly and unambiguously in writing by a designated third party for a specific product). THE SERVICES AND CONTENT ARE PROVIDED BY METACULUS (AND ITS LICENSORS AND SUPPLIERS) ON AN “AS-IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR THAT USE OF THE SERVICES WILL BE UNINTERRUPTED OR ERROR-FREE. SOME STATES DO NOT ALLOW LIMITATIONS ON HOW LONG AN IMPLIED WARRANTY LASTS, SO THE ABOVE LIMITATIONS MAY NOT APPLY TO YOU.

Limitation of Liability. TO THE FULLEST EXTENT ALLOWED BY APPLICABLE LAW, UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY (INCLUDING TORT, CONTRACT, NEGLIGENCE, STRICT LIABILITY, OR OTHERWISE) WILL METACULUS (OR ITS LICENSORS OR SUPPLIERS) BE LIABLE TO YOU OR TO ANY OTHER PERSON FOR (A) ANY INDIRECT, SPECIAL OR INCIDENTAL DAMAGES OF ANY KIND, INCLUDING DAMAGES FOR LOST PROFITS, LOSS OF GOODWILL, WORK STOPPAGE, ACCURACY OF RESULTS, OR COMPUTER FAILURE OR MALFUNCTION, OR (B) ANY AMOUNT, IN THE AGGREGATE, IN EXCESS OF THE GREATER OF (I) $100 OR (II) THE AMOUNTS PAID BY YOU TO METACULUS IN CONNECTION WITH THE SERVICES IN THE TWELVE (12) MONTH PERIOD PRECEDING THIS APPLICABLE CLAIM, OR (C) ANY MATTER BEYOND OUR REASONABLE CONTROL. SOME STATES DO NOT ALLOW THE EXCLUSION OR LIMITATION OF CERTAIN DAMAGES, SO THE ABOVE LIMITATION AND EXCLUSIONS MAY NOT APPLY TO YOU.

Indemnity. To the fullest extent allowed by applicable law, you will defend, indemnify and hold Metaculus and their affiliates, officers, agents, employees, and partners harmless from and against any and all claims, liabilities, damages (actual and consequential), losses and expenses (including attorneys' fees) arising from or in any way related to any third party claims (including from other users) relating to (a) your submissions to the Services including any Content, User Submissions or Competitions, (b) your use of the Services (including any actions taken by a third party using your account), and (c) your violation of these Terms.

Assignment. You may not assign, delegate or transfer these Terms or your rights or obligations hereunder, or your Services account, in any way (by operation of law or otherwise) without Metaculus's prior written consent. We may transfer, assign, or delegate these Terms and our rights and obligations without consent.

About These Terms

These Terms control the relationship between Metaculus and you. They do not create any third party beneficiary rights.

If you do not comply with these Terms, and we don't take action right away, this doesn't mean that we are giving up any rights that we may have (such as taking action in the future).

If it turns out that a particular term is not enforceable, this will not affect any other part of the Terms.

The laws of California, USA, excluding California's conflict of laws rules, will apply to any disputes arising out of or relating to these Terms or the Services. All claims arising out of or relating to these Terms or the Services will be litigated exclusively in the federal or state courts of San Francisco, California, USA, and you and Metaculus consent to personal jurisdiction in those courts.