Difference between revisions of "HFI-Validation"

From Planck Legacy Archive Wiki
Jump to: navigation, search
m (→‎HFI simulations)
m
 
(175 intermediate revisions by 9 users not shown)
Line 1: Line 1:
−
The HFI validation is mostly modular. That is, each part of the pipeline, be it timeline processing, map-making, or any other, validates the results of its work at each step of the processing. In addition, we do additional validation with an eye towards overall system integrity. These are described below.
+
{{DISPLAYTITLE:Overall internal validation}}
  
−
==Expected systematics and tests (bottom-up approach)==
+
The overall internal validation of the frequency maps is performed thanks to several tests:
 +
* difference between the PR2 (2015) and PR3 (2018) frequency maps,
 +
* survey difference maps for the PR2 and the PR3 frequency maps,
 +
* spectra of the PR2 and the PR3 data splits,
 +
* comparison of the FFP10 simulations and the PR3 data.
  
−
{{:HFI-bottom_up}}
+
<br>
 +
<span style="font-size:150%">'''Frequency maps for the PR2 and the PR3 and their difference''' </span>
  
−
==Generic approach to systematics==
+
This table shows the PR2 and PR3 maps and their differences in I, Q, and U. This table is complementary of the figure in {{PlanckPapers|planck2016-l03}} (see detailled explanations there).
  
−
While we track and try to limit the individual effects listed above, and we do not believe there are other large effects which might compromise the data, we test this using a suite of general difference tests. As an example, the first and second years of Planck observations used almost exactly the same scanning pattern (they differed by one arc-minute at the Ecliptic plane). By differencing them, the fixed sky signal is almost completely removed, and we are left with only time variable signals, such as any gain variations and, of course, the statistical noise.  
+
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
 +
|+ '''Comparaison of PR2 and PR3 I, Q and U maps and their difference. '''
 +
|- bgcolor="ffdead"
 +
!
 +
!colspan="3"| PR2 frequency maps
 +
!colspan="3"| PR3 frequency maps
 +
!colspan="3"| difference
 +
|-
 +
!
 +
!I
 +
!Q
 +
!U
 +
!I
 +
!Q
 +
!U
 +
!I
 +
!Q
 +
!U
 +
|-
 +
| 100 GHz
 +
|[[File:100GHz_DX11_I.pdf.pdf|100px]]
 +
|[[File:100GHz_DX11_Q.pdf.pdf|100px]]
 +
|[[File:100GHz_DX11_U.pdf.pdf|100px]]
 +
|[[File:100GHz_I.pdf|100px]]
 +
|[[File:100GHz_Q.pdf|100px]]
 +
|[[File:100GHz_U.pdf|100px]]
 +
|[[File:100GHz_diff_I.pdf.pdf|100px]]
 +
|[[File:100GHz_diff_Q.pdf.pdf|100px]]
 +
|[[File:100GHz_diff_U.pdf.pdf|100px]]
 +
|-
 +
| 143 GHz
 +
|[[File:143GHz_DX11_I.pdf.pdf|100px]]
 +
|[[File:143GHz_DX11_Q.pdf.pdf|100px]]
 +
|[[File:143GHz_DX11_U.pdf.pdf|100px]]
 +
|[[File:143GHz_I.pdf|100px]]
 +
|[[File:143GHz_Q.pdf|100px]]
 +
|[[File:143GHz_U.pdf|100px]]
 +
|[[File:143GHz_diff_I.pdf.pdf|100px]]
 +
|[[File:143GHz_diff_Q.pdf.pdf|100px]]
 +
|[[File:143GHz_diff_U.pdf.pdf|100px]]
 +
|-
 +
| 217 GHz
 +
|[[File:217GHz_DX11_I.pdf.pdf|100px]]
 +
|[[File:217GHz_DX11_Q.pdf.pdf|100px]]
 +
|[[File:217GHz_DX11_U.pdf.pdf|100px]]
 +
|[[File:217GHz_I.pdf|100px]]
 +
|[[File:217GHz_Q.pdf|100px]]
 +
|[[File:217GHz_U.pdf|100px]]
 +
|[[File:217GHz_diff_I.pdf.pdf|100px]]
 +
|[[File:217GHz_diff_Q.pdf.pdf|100px]]
 +
|[[File:217GHz_diff_U.pdf.pdf|100px]]
 +
|-
 +
| 353 GHz
 +
|[[File:353GHz_DX11_I.pdf.pdf|100px]]
 +
|[[File:353GHz_DX11_Q.pdf.pdf|100px]]
 +
|[[File:353GHz_DX11_U.pdf.pdf|100px]]
 +
|[[File:353GHz_I.pdf|100px]]
 +
|[[File:353GHz_Q.pdf|100px]]
 +
|[[File:353GHz_U.pdf|100px]]
 +
|[[File:353GHz_diff_I.pdf.pdf|100px]]
 +
|[[File:353GHz_diff_Q.pdf.pdf|100px]]
 +
|[[File:353GHz_diff_U.pdf.pdf|100px]]
 +
|-
 +
| 545 GHz
 +
|[[File:545GHz_DX11_I.pdf.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:545GHz_I.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:545GHz_diff_I.pdf.pdf|100px]]
 +
| .
 +
| .
 +
|-
 +
| 857 GHz
 +
|[[File:857GHz_DX11_I.pdf.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:857GHz_I.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:857GHz_diff_I.pdf.pdf|100px]]
 +
| .
 +
| .
 +
|}
  
−
In addition, while Planck scans the sky twice a year, during the first six months (or survey) and the second six months (the second survey), the orientations of the scans and optics are actually different. Thus, by forming a difference between these two surveys, in addition to similar sensitivity to the time-variable signals seen in the yearly test, the survey difference also tests our understanding and sensitivity to scan-dependent noise such as time constant and beam asymmetries.
 
  
−
These tests use the <tt>Yardstick</tt> simulations below and culminate in the "Probabilities to Exceed" tests just after.
+
<br>
 +
<span style="font-size:150%">'''Survey difference maps for the PR2 and the PR3 data''' </span>
  
−
==HFI simulations==
+
This table shows the PR2 and PR3 survey difference maps ((S1+S3)-(S2+S4))in I, Q, and U. This table is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).
  
−
The '<tt>Yardstick</tt>' simulations allows gauging various effects to see whether they need be included in monte-carlo to describe data. It also allows gauging the significance of validation tests on data (e.g. can null test can be described by the model?). They are completed by dedicated '<tt>Desire</tt>' simulations (<tt>Desire</tt> stands for DEtector SImulated REsponse), as well as Monte-Carlo simulations of the Beams determination to determine their uncertainty.
+
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
 +
|+ '''Comparaison of PR2 and PR3 I, Q and U survey difference maps.'''
 +
|- bgcolor="ffdead"
 +
!
 +
!colspan="3"| PR2 survey difference maps
 +
!colspan="3"| PR3 survey difference maps
 +
|-
 +
!
 +
!I
 +
!Q
 +
!U
 +
!I
 +
!Q
 +
!U
 +
|-
 +
| 100 GHz
 +
|[[File:100GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:100GHz_DX11_surveyS1S3_S2S4_Q.pdf.pdf|100px]]
 +
|[[File:100GHz_DX11_surveyS1S3_S2S4_U.pdf.pdf|100px]]
 +
|[[File:100GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:100GHz_RD12RC4_surveyS1S3_S2S4_Q.pdf|100px]]
 +
|[[File:100GHz_RD12RC4_surveyS1S3_S2S4_U.pdf|100px]]
 +
|-
 +
| 143 GHz
 +
|[[File:143GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:143GHz_DX11_surveyS1S3_S2S4_Q.pdf.pdf|100px]]
 +
|[[File:143GHz_DX11_surveyS1S3_S2S4_U.pdf.pdf|100px]]
 +
|[[File:143GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:143GHz_RD12RC4_surveyS1S3_S2S4_Q.pdf|100px]]
 +
|[[File:143GHz_RD12RC4_surveyS1S3_S2S4_U.pdf|100px]]
 +
|-
 +
| 217 GHz
 +
|[[File:217GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:217GHz_DX11_surveyS1S3_S2S4_Q.pdf.pdf|100px]]
 +
|[[File:217GHz_DX11_surveyS1S3_S2S4_U.pdf.pdf|100px]]
 +
|[[File:217GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:217GHz_RD12RC4_surveyS1S3_S2S4_Q.pdf|100px]]
 +
|[[File:217GHz_RD12RC4_surveyS1S3_S2S4_U.pdf|100px]]
 +
|-
 +
| 353 GHz
 +
|[[File:353GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:353GHz_DX11_surveyS1S3_S2S4_Q.pdf.pdf|100px]]
 +
|[[File:353GHz_DX11_surveyS1S3_S2S4_U.pdf.pdf|100px]]
 +
|[[File:353GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
|[[File:353GHz_RD12RC4_surveyS1S3_S2S4_Q.pdf|100px]]
 +
|[[File:353GHz_RD12RC4_surveyS1S3_S2S4_U.pdf|100px]]
 +
|-
 +
| 545 GHz
 +
|[[File:545GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:545GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
| .
 +
| .
 +
|-
 +
| 857 GHz
 +
|[[File:857GHz_DX11_surveyS1S3_S2S4_I.pdf|100px]]
 +
| .
 +
| .
 +
|[[File:857GHz_RD12RC4_surveyS1S3_S2S4_I.pdf|100px]]
 +
| .
 +
| .
 +
|}
  
−
[[Image:HFI-sims.png|HFI simulations chain(s)]]
 
−
 
−
===<tt>Yardstick</tt> simulations===
 
  
−
The <tt>Yardstick</tt> V3.0 characterizes the DX9 data which is the basis of the data release. It goes through the following steps:
+
<br>
 +
<span style="font-size:150%">'''Spectra of the PR2 and the PR3 data splits''' </span>
  
−
#The input maps are computed using the Planck Sky Model, taking the RIMO bandpasses as input.
+
This figure shows the ''EE'' and ''BB'' spectra of the PR2 and PR3 detset, half-mission and rings (for PR3 only) maps at 100, 143, 217, and 353 GHz. The auto-spectra of the difference maps and the cross-spectra between the maps are shown. The sky fraction used here is 43 %. The bins are: bin=1 for <math>2\leq\ell<30</math>; bin=5 for <math>30\leq\ell<50</math>; bin=10 for <math>50\leq\ell<160</math>; bin=20 for <math>160\leq\ell<1000</math>; and bin=100 for <math>\ell>1000</math>. This figure is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).  
−
#The <tt>LevelS</tt> is used to project input maps on timeline using the RIMO(B-Spline) scanning beam and the DX9 pointing (called ptcor6). The real pointing is affected by the aberration that is corrected by map-making. The <tt>Yardstick</tt> does not simulate aberration. Finally, the difference between the projected pointing from simulation and from DX9 is equal to the aberration.
 
−
#The simulated noise timelines, that are added to the projected signal, have the same spectrum (low and high frequency) than the DX9 noise. For the<tt>yardstick</tt> V3.0 Althoough detectable, no correlation in time or between detectors have been simulated.
 
−
#The simulation map making step use the DX9 sample flags.
 
−
#For the low frequencies (100, 143, 217, 353), the <tt>yardstick</tt> output are calibrated using the same mechanism (e.g. dipole fitting) than DX9. This calibration step is not perfromed for higher frequency (545, 857) which use a differnt principle
 
−
#The Official map making is run on those timelines using the same parameters than for real data.
 
−
A <tt>yardstick</tt> production is composed of
 
−
* all survey map (1,2 and nominal),
 
−
* all detector Detsets (from individual detectors to full channel maps).
 
−
The <tt>Yardstick</tt> V3.0 is based on 5 noise iterations for each map realization.
 
  
−
NB1: the <tt>Yardstick</tt> product is also the validating set for other implementations which are not using the HFI DPC production codes, an exemple of which are the so-called <tt>FFP</tt> simulations, where FFP stands for Full Focal Plane and are done in common by HFI & LFI. This is further described in [[HL-sims]]
+
<center>
 +
[[File:cl_fsky43_DX11_RD12RC4_3000_oddeven_multiplot.pdf|500px]]
 +
</center>
  
−
NB2: A dedicated version has been used for Monte-Carlo simulations of the beams determination, or <tt>MCB</tt>. See [[Pointing&Beams#Simulations_and_errors]]
 
  
−
===<tt>Desire</tt> simulations===
+
<br>
 +
<span style="font-size:150%">'''Comparison of the FFP10 simulated noise and systematic residuals and the PR3 data''' </span>
  
−
The <tt>yarstick</tt> simulations do not go through actual TOI processing, but rather simulate the TOI expected at the end of the TOI processing step, as in input to the map-making step. Some effects cannot be easily simulated like this. The <tt>Desire</tt> simulations are used in conjunction with the actual TOI processing, to analyse e.g., the impact of 4K line residuals.
+
This figure shows the noise and systematic residuals in ''TT'', ''EE'', ''BB'', and ''EB'' spectra, at the three CMB frequencies, for difference maps of the ring (red) and half-mission (blue) null tests binned by <math>\Delta \ell =10</math>. Data spectra are represented by thick lines, and the averages of simulations by thin black lines. For the simulations, we show the 16 % and 84 % quantiles of the distribution with the same colours. This figure is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).
  
−
==Simulations versus data==
+
<center>
 +
[[File:newpte.pdf|500px]]
 +
</center>
  
−
The significance of various difference tests perfromed on data can be assessed in particular by comparing them with <tt>Yardstick</tt> realisations.
 
  
−
<tt>Yardstick</tt> production contains sky (generated with <tt>LevelS</tt> starting from <tt>PSM<tt> V1.77) and noise timeline realisations proceeded with the official map making. DX9 production was regenerated with the same code in order to get rid of possible differences that might appear for not running the official pipeline in the same conditions.
 
  
−
We compare statistical properties of the cross spectra of null test maps for the 100, 143, 217, 353 GHz channels. Null test maps can either be survey null test or half focal plane null test, each of which having a specific goal : # survey1-survey2 (S1-S2) aim at isolating transfer function or pointing issues, while
+
==References==
−
# half focal plane null tests enable to focus on beam issues.
 
−
Comparing cross spectra we isolate systematic effects from the noise, and we
 
−
can check whether they are properly simulated or need to. Spectra are computed with <tt>spice</tt> masking either DX9 point sources or simulated point sources, and masking the galactic plane with several mask width, the sky fraction from which spectra are computed are around 30%, 60% and 80%.
 
  
−
DX9 and the Y3.0 realisations are binned. For each bin we compute the statistical parameters (mean and variance) of the <tt>Yardstick</tt> distribution. The following figure is a typical example of a consistency test, it shows the differences between Y3.0 mean and DX9 considering the standard deviation of the yardstick. We also indicate chi square values, which are computed within larger bin : [0,20], [20,400], [400,1000][1000,2000], [2000, 3000], using the ratio between (DX9-Y3.0 mean)<sup>2</sup> and Y3.0 variance within each bin. This binned chi-square is only indicative: it may not be always significant, since DX9 variations sometimes disappear as we average them in a bin, the mean is then at the same scale as the yardstick one.
+
<References />
 +
  
−
[[File:DX9_Y3_consistency.png | 500px  | center | thumb | '''Example of consistency test for 143 survey null test maps.''']]
 
  
−
Here will be a linlk to a big pdf file with all those plots, or a visualisation page.
+
[[Category:HFI data processing|006]]

Latest revision as of 08:40, 3 July 2018


The overall internal validation of the frequency maps is performed thanks to several tests:

  • difference between the PR2 (2015) and PR3 (2018) frequency maps,
  • survey difference maps for the PR2 and the PR3 frequency maps,
  • spectra of the PR2 and the PR3 data splits,
  • comparison of the FFP10 simulations and the PR3 data.


Frequency maps for the PR2 and the PR3 and their difference

This table shows the PR2 and PR3 maps and their differences in I, Q, and U. This table is complementary of the figure in Planck-2020-A3[1] (see detailled explanations there).

Comparaison of PR2 and PR3 I, Q and U maps and their difference.
PR2 frequency maps PR3 frequency maps difference
I Q U I Q U I Q U
100 GHz 100GHz DX11 I.pdf.pdf 100GHz DX11 Q.pdf.pdf 100GHz DX11 U.pdf.pdf 100GHz I.pdf 100GHz Q.pdf 100GHz U.pdf 100GHz diff I.pdf.pdf 100GHz diff Q.pdf.pdf 100GHz diff U.pdf.pdf
143 GHz 143GHz DX11 I.pdf.pdf 143GHz DX11 Q.pdf.pdf 143GHz DX11 U.pdf.pdf 143GHz I.pdf 143GHz Q.pdf 143GHz U.pdf 143GHz diff I.pdf.pdf 143GHz diff Q.pdf.pdf 143GHz diff U.pdf.pdf
217 GHz 217GHz DX11 I.pdf.pdf 217GHz DX11 Q.pdf.pdf 217GHz DX11 U.pdf.pdf 217GHz I.pdf 217GHz Q.pdf 217GHz U.pdf 217GHz diff I.pdf.pdf 217GHz diff Q.pdf.pdf 217GHz diff U.pdf.pdf
353 GHz 353GHz DX11 I.pdf.pdf 353GHz DX11 Q.pdf.pdf 353GHz DX11 U.pdf.pdf 353GHz I.pdf 353GHz Q.pdf 353GHz U.pdf 353GHz diff I.pdf.pdf 353GHz diff Q.pdf.pdf 353GHz diff U.pdf.pdf
545 GHz 545GHz DX11 I.pdf.pdf . . 545GHz I.pdf . . 545GHz diff I.pdf.pdf . .
857 GHz 857GHz DX11 I.pdf.pdf . . 857GHz I.pdf . . 857GHz diff I.pdf.pdf . .



Survey difference maps for the PR2 and the PR3 data

This table shows the PR2 and PR3 survey difference maps ((S1+S3)-(S2+S4))in I, Q, and U. This table is taken from Planck-2020-A3[1] (see detailled explanations there).

Comparaison of PR2 and PR3 I, Q and U survey difference maps.
PR2 survey difference maps PR3 survey difference maps
I Q U I Q U
100 GHz 100GHz DX11 surveyS1S3 S2S4 I.pdf 100GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 100GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 100GHz RD12RC4 surveyS1S3 S2S4 I.pdf 100GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 100GHz RD12RC4 surveyS1S3 S2S4 U.pdf
143 GHz 143GHz DX11 surveyS1S3 S2S4 I.pdf 143GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 143GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 143GHz RD12RC4 surveyS1S3 S2S4 I.pdf 143GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 143GHz RD12RC4 surveyS1S3 S2S4 U.pdf
217 GHz 217GHz DX11 surveyS1S3 S2S4 I.pdf 217GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 217GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 217GHz RD12RC4 surveyS1S3 S2S4 I.pdf 217GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 217GHz RD12RC4 surveyS1S3 S2S4 U.pdf
353 GHz 353GHz DX11 surveyS1S3 S2S4 I.pdf 353GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 353GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 353GHz RD12RC4 surveyS1S3 S2S4 I.pdf 353GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 353GHz RD12RC4 surveyS1S3 S2S4 U.pdf
545 GHz 545GHz DX11 surveyS1S3 S2S4 I.pdf . . 545GHz RD12RC4 surveyS1S3 S2S4 I.pdf . .
857 GHz 857GHz DX11 surveyS1S3 S2S4 I.pdf . . 857GHz RD12RC4 surveyS1S3 S2S4 I.pdf . .



Spectra of the PR2 and the PR3 data splits

This figure shows the EE and BB spectra of the PR2 and PR3 detset, half-mission and rings (for PR3 only) maps at 100, 143, 217, and 353 GHz. The auto-spectra of the difference maps and the cross-spectra between the maps are shown. The sky fraction used here is 43 %. The bins are: bin=1 for [math]2\leq\ell\lt 30[/math]; bin=5 for [math]30\leq\ell\lt 50[/math]; bin=10 for [math]50\leq\ell\lt 160[/math]; bin=20 for [math]160\leq\ell\lt 1000[/math]; and bin=100 for [math]\ell\gt 1000[/math]. This figure is taken from Planck-2020-A3[1] (see detailled explanations there).

Cl fsky43 DX11 RD12RC4 3000 oddeven multiplot.pdf



Comparison of the FFP10 simulated noise and systematic residuals and the PR3 data

This figure shows the noise and systematic residuals in TT, EE, BB, and EB spectra, at the three CMB frequencies, for difference maps of the ring (red) and half-mission (blue) null tests binned by [math]\Delta \ell =10[/math]. Data spectra are represented by thick lines, and the averages of simulations by thin black lines. For the simulations, we show the 16 % and 84 % quantiles of the distribution with the same colours. This figure is taken from Planck-2020-A3[1] (see detailled explanations there).

Newpte.pdf


References[edit]

  1. ↑ 1.01.11.21.3 Planck 2018 results. III. High Frequency Instrument data processing and frequency maps, Planck Collaboration, 2020, A&A, 641, A3.

Cosmic Microwave background

Retrieved from ‘https://wiki.cosmos.esa.int/planck-legacy-archive/index.php?title=HFI-Validation&oldid=13928’