WFIRST

WFIRST Simulations


Wide-Field Infrared Survey Telescope

CGI Exoplanet Imaging Data Challenge

Welcome to the next community data challenge for the WFIRST Coronagraph Instrument (CGI)! The WFIRST mission is currently in Phase B with a launch date set to ~late 2025, and teams are actively working towards the Preliminary Design Review for CGI. The guaranteed "Tech(nology) Demo(nstrator)" phase is currently set for ~3 months of observing during the first 1.5 years of the mission. Thereafter, NASA envisions a Participating Scientist Program (PSP) for community involvement. CGI will be the first instrument to approach the contrast and angular separation parameters needed to directly image mature gas giant exoplanets in reflected starlight and much more.

Data challenges aim at engaging the community, to "broaden and deepen" its knowledge. The Data Challenge will kick off on October 16th-17th 2019 at the Flatiron Institute in New York City followed by a special session on Sunday October 29th in Tokyo prior to the In the Spirit of Lyot 2019 conference. It will close on December 20, 2019. Participating teams will thus have two months to answer several questions, ranging from trivial to not-so-trivial, and we plan to advertise results at the Exoplanets III conference in Heidelberg, July 27-31, 2020. More information about the CGI Exoplanet Imaging Data Challenge including a preregistration rink is available.

This time, the challenge will focus on point-source extraction (angular and reference differential imaging) and astrometry (orbital fitting). We wish to engage as many participants as possible, and for these reasons we have organized "hackathon" mini-workshops to rehearse and improve our material before launching the real data challenge.

  • Hackathon #1 at STScI: March 18-19, 2019 -- Completed
  • Hackathon #2 at IPAC: June 24-25, 2019 -- Completed

The challenge is particularly well suited for students and junior researchers worldwide interested in high contrast imaging of exoplanetary systems. It is a fantastic opportunity to learn about the WFIRST CGI data format (EMCCD detector), specificities, and observing scenarios, and to make yourself known to the CGI community, Science Investigation Teams (SIT), future PSP, Project Office, etc.



Example of three epochs (two Hybrid Lyot Coronagraph, one Starshade) for a Data Challenge 2 rehearsal and hackathon.


Some of the problems that participants will face:

  • Extract hidden objects around a nearby star with PSF subtraction and eventually post-processing
  • Discriminate injected planet(s) from eventual background, noise, or contamination source(s) using parallax and proper motion information
  • Find orbital solution(s) combining:
    • Several astrometric epochs throughout several years of the mission including both Hybrid Lyot and Starshade coronagraphs
    • Known radial velocity measurements
  • Determine physical parameters of the found exoplanet(s): phase(s), albedo(s)



Hackathon number 1 at STScI, March 2019


For updated information check the data challenge webpage.

To prepare scientists to participate, two special events will be held: a tutorial and hack day at the Flatiron Institute in New York City on October 16&17, and a tutorial in Tokyo on October 20, immediately prior to the In the Spirit of Lyot 2019 conference.

The official Data Challenge start date is October 20, at which point a new data set including the first epochs of CGI imaging and RV precursor data will be released. Findings from this first data release will be due on November 20, at which time respondents will be given CGI and Starshade images for the remaining epochs. The Data Challenge will close on December 20th, 2019.

Sign up for the 2019 Data Challenge here.

If you have questions, please forward them to PI Margaret Turnbull (SETI) or Julien Girard (STScI).



Microlensing Data Challenge #1 (now closed)

What was the Challenge?

WFIRST is expected to detect thousands of microlensing events, including hundreds of planetary ones. Traditionally, interesting lightcurves were modeled one-by-one given that the most exciting events were relatively rare. Such interactive modeling will not be possible as the data set grows, so to fully exploit this dataset, analysis techniques need to be upgraded.

To stimulate research in this area, there will be a series of microlensing data challenges, the first of which is based around a large set of simulated WFIRST lightcurves. This dataset has now been released, with a submission deadline of Oct 31, 2018.

For more information, please visit http://microlensing-source.org/data-challenge/ .

There is also a python notebook tutorial to provide a starting point for newcomers wishing to take part.



What were the Results?

A total of 293 light curves in filters Z087 and W149 were simulated by Matthew Penny, and this included 74 single lenses (including free floating planet candidates), 83 binary star lenses, 43 planetary binary lenses, and 93 cataclysmic variables. The simulated light curves mimicked the cadence, length, and noise properties of the multi-year WFIRST Bulge survey.


A sample lightcurve mimicking the expected cadence and noise characteristics of WFIRST microlensing observations

Four teams entered the challenge, including 16 participants in total, seven of whom were newcomers to the microlensing field. Teams were tasked with classifying each light curve and providing fits to the model parameters of each. Teams were evaluated on the accuracy of their fitted parameters, the efficiency and scalability of their software and modeling approaches, innovations they brought to the table, and the extent to which newcomers participated.

All teams used publicly available software, and progress was made on the issue of scalability, although there is still room for improvement. Some new approaches to classification and detection were developed but are still in their early stages. Results from each of the teams showed that, when microlensing events were properly classified, parameters were also accurately derived, modulo some known weaknesses such as a tendency to overestimate the impact parameter. However, the problem of classification is non-trivial, particularly for subtle anomalies in the light curves. Formal benchmarking, which was not attempted in this challenge due to logistical reasons, would have provided a more meaningful comparison between teams. More details can be found in this slide package.

Written feedback was sent to each team, and a paper documenting the challenge and its results is now being written. A second data challenge, building on the lessons learned here, will soon follow.



CGI Exoplanet Spectral Imaging Data Challenge (now closed)

What was the Challenge?

Welcome to the inaugural CGI Exoplanet Data Challenge! The WFIRST mission is currently in Phase A, during which time the science and instrument performance requirements will be defined for exoplanet imaging and spectroscopy. In order to provide the project with the best possible inputs before the end of Phase A in 2017, we are seeking participation from teams with spectral retrieval expertise through the data challenge.

The Challenge will run from August 2016 to March 2017. The 2016 Challenge consists of a blind spectral retrieval exercise using simulated, extracted spectra for several known RV and/or hypothetical discovery exoplanets. The spectra will NOT need to be extracted from simulated IFS data. Instead, we will explore the impact of signal-to-noise ratio and spectral resolution on the detection/measurement of atmospheric abundances and other planet properties. Even with that relatively simple goal, we expect the Challenge to be non-trivial!



Sample spectra from the Data Challenge. Shown are simulated spectra for three of the four exoplanets at the highest signal-to-noise and resolving power (top row) and at the lowest signal-to-noise and resolving power (bottom row).


Incentive to Participate: While defining the first space-borne exoplanet imaging mission is hopefully its own compelling reason for doing this, to make this a little more fun the CGI Exoplanet Spectral Imaging Data Challenge Science Investigation Team is offering travel expenses and registration costs for one person on each team that fully completes the Challenge (all four planets, all SNR and R values, all requested retrieval outputs) to attend the 2017 WFIRST Science Meeting, or another exoplanets meeting of his/her choice (up to $2000).

Participation in the Challenge is contingent upon acceptance of terms which will be included in the invitation email.

If you wish to participate, please register and you will be sent an invitation.

If you have questions, please forward them to Margaret Turnbull and David Ciardi through the "Contact" link above.

We look forward to working with you this Fall!

What were the Results?

The CGI Exoplanet Spectral Imaging Data Challenge is a working group within the SIT "Harnessing the Power of the WFIRST-Coronagraph" (PI, Margaret Turnbull). The goal is to provide a quantitative and qualitative feedback to the teams defining the specifications and requirements of the WFIRST coronagraph by blindly retrieving realistic estimates of the expected planetary and atmospheric properties that can be recovered in the WFIRST mission.

Renyu Hu, Tyler Robinson, and Jake Lustig-Yaeger produced simulated data representative of the Integral Field Spectrograph (IFS) instrument of WFIRST. The simulations covered different types of planets (Hot giants, Sub-Neptunes, and rocky ones). They included different instrumental modes of the WFIRST IFS and covered a representative range of different signal-to-noise scenarios. They had different concentrations of CH4, NH3, an H2O in their atmospheres, as well as different temperatures and masses. In this Data Challenge, we considered two giant planets, one sub-Neptune, and two rocky planets.

There were numerous participants from institutions in Asia, Europe, and the US, with several members outside the WFIRST community, and we held regular telecon meetings to share our results.

One important outcome from the Data Challenge was the cross-comparison of the atmospheric modeling among the different teams, significantly increasing the agreement among them, while still leaving room for a variety of atmospheric models. Some teams were able to recover atmospheric parameters.

Two teams, NEMESIS (P. Irwin, J. Eberhardt, and R. Garland) and the team led by M. Marley, R. Lupu, and M. Nayak, were able to recover the atmospheric parameters for the first two giant planets including a variety of models with clouds and hazes. Our major conclusion is that CH4 concentration was well recovered for the WFIRST data. NH3 and H2O recovery was acceptable. The latter was highly dependent on the signal-to-noise ratio and absorption tables used to derive the results. Finally, fundamental parameters, such as the mass of the planet, had a high statistical precision but were somewhat biased depending on the atmospheric model.



The figure above shows the results from two teams. Left: Data and retrieved spectra (credit M. Nayak). Right: Recovered atmospheric parameters for one gas giant (credit: R. Garland).