NeurIPS 2019

Getting Started in Python

#pip3 install --user --upgrade

import numpy as np

from sys import exit, exc_info, argv

import random

from ushiriki_policy_engine_library.NeurIPS19ChallengeEnvironment import ChallengeEnvironment

from ushiriki_policy_engine_library.EvaluateSubmission import EvaluateAugmentedChallengeSubmission,  EvaluateChallengeSubmission 

class ChallengeEnvironment1(ChallengeEnvironment):

    def __init__(self):

# Please replace the Agent userID code with the code you received after joining the competition.


        ChallengeEnvironment.__init__(self,baseuri="", userID="REPLACE_WITH_ID_RECEIVED" , experimentCount=2000, token="e4e2db39-5daa-48e5-84b4-2460d2844a10" )# for the demo proxy server

class CustomAgent:

    def __init__(self, environment):

        self.environment = environment

    def generate(self):

        best_policy = None

        best_reward = -float('Inf')

        candidates = []


            # Agents should make use of 20 episodes in each training run, if making sequential decisions

            for i in range(20):


                policy = {}

                for j in range(self.environment.policyDimension): #episode length




            rewards = self.environment.evaluatePolicy(candidates)

            best_policy = candidates[np.argmax(rewards)]

            best_reward = rewards[np.argmax(rewards)]


        except (KeyboardInterrupt, SystemExit):


        return best_policy, best_reward

eval = EvaluateAugmentedChallengeSubmission(ChallengeEnvironment1, CustomAgent, "test_stop.csv")


I want to know more about Malaria? 


What are ITNs and IRS? 


Insecticide Treated Nets (ITNs) and Indoor Residual Spraying (IRS) are among the most studied interventions in modelling malaria transmission 


Where can I find more information about Reinforcement Learning? 


It is a subject covered in terms of our material in the tutorial but you will find extensive reading materials and many useful blogs based on your level and interest. 


What malaria setting is being modelled? 


For the purposes of this challenge we have developed our own model for an abstract location, to avoid any distraction outside of the NeurIPS environment. And preventing any advantage which could be provided from background material.

Live Malaria Challenge

Malaria is thought to have had the greatest disease burden throughout human history, while it continues to pose a significant but disproportionate global health burden. With 50% of the world’s population at risk of malaria infection.  Sub Saharan Africa is most affected, with 90% of all cases.

Over the course of this live competition we are looking for participants to apply machine learning tools to determine novel solutions which could impact malaria policy in Sub Saharan Africa. Specifically, how should combinations of interventions which control the transmission, prevalence and health outcomes of malaria infection, be distributed in a simulated human population.

This challenge has been framed as a Reinforcement Learning problem, participants are expected to submit high performing solutions to the sequential decision making task. For this competition, actions receive stochastic and delayed rewards, which are resource constrained by both the monetary and computational costs associated with implementing and observing the impact of an action.

Submissions are encouraged from participants who may not have previous experience in reinforcement learning problems, reading through the materials we hope you may see the necessity for contributions to inform decision making for this complex real-world problem.


Code must be submitted through the competition platform by the deadline 1200 noon PST Friday 13th December 2019. The top 3 participants on the Leaderboard at this time have the opportunity to present (5 mins) at the Machine Learning Competitions workshop. All participants may attend and are invited to contribute towards Workshop Proceedings.

Using your NeurIPS Registration ID is required to confirm registration and participation in the live Challenge.

If the winners wish to publish results we request that the Competition be appropriately referenced and code publicly available under an OSI-approved license such as, for instance, Apache 2.0, MIT or BSD-like license.



To receive announcements and be informed of any change in rules, the participants must provide a valid email to the challenge platform along with their NeurIPS registration ID.

Conditions of participation:

Participation requires complying with the rules of the Competition. Prize eligibility is restricted by US government export regulations. The organizers, sponsors, their students, close family members (parents, sibling, spouse or children) and household members, as well as any person having had access to the truth values or to any information about the data or the Competition design giving him (or her) an unfair advantage are excluded from participation. A disqualified person may submit one or several entries in the Competition and request to have them evaluated, provided that they notify the organizers of their conflict of interest. If a disqualified person submits an entry, this entry will not be part of the final ranking and does not qualify for prizes. The participants should be aware that the organizers reserve the right to evaluate for scientific purposes any entry made in the challenge, whether or not it qualifies for prizes.


All participants will be invited to attend a workshop organized in conjunction with NeurIPS and contribute to the proceedings.


The participants must register to the platform and provide a valid email address along with Conference Regisration ID. Team acknowledgement or submission will not be supported for this competition due to it’s live, on-site nature.


The participants who do not present their results at the workshop can elect to remain anonymous by using a pseudonym. Their results will be published on the leaderboard under that pseudonym, and their real name will remain confidential. However, the participants must disclose their real identity to the organizers to receive further contact and acknowledgement in published material. 

Submission Expectation:

Your submission is based on 10 instantiations of the environment object

Each environment has a limit of 2000 evaluations, with an episode requiring 5 evaluations

Your submission should include submission of a learnt policy and it's evaluation, in the provided .csv format

Best practices and expectations should refer to the example and tutorial code



Rank Team Score Count Submitted Date
1 host 23.75402344 1 Dec. 9, 2019, 7:39 a.m.

Data License

In-App Messaging
quanjun chen
The University of Tokyo

Note: This competition is hosted by us in collaboration with IBM and University of Oxford. For questions about the content for this competition, please contact Competition Host . For questions regarding the website features please contact