Difference between revisions of "HFI-Validation"

From Planck Legacy Archive Wiki
Jump to: navigation, search
m
 
(14 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
{{DISPLAYTITLE:Overall internal validation}}
 
{{DISPLAYTITLE:Overall internal validation}}
  
The overall internal validation of the frequency maps is seen thanks to several tests:
+
The overall internal validation of the frequency maps is performed thanks to several tests:
* difference between the PR2 and PR3 frequency maps,
+
* difference between the PR2 (2015) and PR3 (2018) frequency maps,
 
* survey difference maps for the PR2 and the PR3 frequency maps,
 
* survey difference maps for the PR2 and the PR3 frequency maps,
*  
+
* spectra of the PR2 and the PR3 data splits,
*
+
* comparison of the FFP10 simulations and the PR3 data.
  
 +
<br>
 
<span style="font-size:150%">'''Frequency maps for the PR2 and the PR3 and their difference''' </span>
 
<span style="font-size:150%">'''Frequency maps for the PR2 and the PR3 and their difference''' </span>
  
This table shows the PR2 (2015) and PR3 (2017) maps and their differences in I, Q, and U. This table is complementary of Figure 11 of {{PlanckPapers|planck2016-l03}} (see detailled explanations there).  
+
This table shows the PR2 and PR3 maps and their differences in I, Q, and U. This table is complementary of the figure in {{PlanckPapers|planck2016-l03}} (see detailled explanations there).  
  
 
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
 
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
|+ '''Comparaison of 2015 and 2017 I, Q and U maps and their difference. '''
+
|+ '''Comparaison of PR2 and PR3 I, Q and U maps and their difference. '''
 
|- bgcolor="ffdead"
 
|- bgcolor="ffdead"
 
!
 
!
!colspan="3"| 2015 frequency maps
+
!colspan="3"| PR2 frequency maps
!colspan="3"| 2017 frequency maps
+
!colspan="3"| PR3 frequency maps
 
!colspan="3"| difference
 
!colspan="3"| difference
 
|-
 
|-
Line 99: Line 100:
  
 
<br>
 
<br>
 
 
<span style="font-size:150%">'''Survey difference maps for the PR2 and the PR3 data''' </span>
 
<span style="font-size:150%">'''Survey difference maps for the PR2 and the PR3 data''' </span>
  
This table shows the PR2 (2015) and PR3 (2017) survey difference maps in I, Q, and U. This table is taken from Figure 12 of {{PlanckPapers|planck2016-l03}} (see detailled explanations there).  
+
This table shows the PR2 and PR3 survey difference maps ((S1+S3)-(S2+S4))in I, Q, and U. This table is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).  
 
 
LVLV : a remplir !!!!!
 
  
 
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
 
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:centert" width=800px
|+ '''Comparaison of 2015 and 2017 I, Q and U survey difference maps.'''
+
|+ '''Comparaison of PR2 and PR3 I, Q and U survey difference maps.'''
 
|- bgcolor="ffdead"
 
|- bgcolor="ffdead"
 
!
 
!
!colspan="3"| 2015 survey difference maps
+
!colspan="3"| PR2 survey difference maps
!colspan="3"| 2017 survey difference maps
+
!colspan="3"| PR3 survey difference maps
 
|-
 
|-
 
!
 
!
Line 172: Line 170:
  
 
<br>
 
<br>
 +
<span style="font-size:150%">'''Spectra of the PR2 and the PR3 data splits''' </span>
  
 +
This figure shows the ''EE'' and ''BB'' spectra of the PR2 and PR3 detset, half-mission and rings (for PR3 only) maps at 100, 143, 217, and 353 GHz. The auto-spectra of the difference maps and the cross-spectra between the maps are shown. The sky fraction used here is 43 %. The bins are: bin=1 for <math>2\leq\ell<30</math>; bin=5 for <math>30\leq\ell<50</math>; bin=10 for <math>50\leq\ell<160</math>; bin=20 for <math>160\leq\ell<1000</math>; and bin=100 for <math>\ell>1000</math>. This figure is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).
  
 +
<center>
 +
[[File:cl_fsky43_DX11_RD12RC4_3000_oddeven_multiplot.pdf|500px]]
 +
</center>
  
  
 +
<br>
 +
<span style="font-size:150%">'''Comparison of the FFP10 simulated noise and systematic residuals and the PR3 data''' </span>
  
HFI validation is mostly modular. In other words, each part of the pipeline, e.g., timeline processing or mapmaking, has the results of its work validated at each step of the processing, looking specifically for known issues. In addition, we perform additional validation with an eye towards overall system integrity, by looking at generic differences between sets of maps, in which most problems will become apparent (whether known or not). Both these approaches are described below.
+
This figure shows the noise and systematic residuals in ''TT'', ''EE'', ''BB'', and ''EB'' spectra, at the three CMB frequencies, for difference maps of the ring (red) and half-mission (blue) null tests binned by <math>\Delta \ell =10</math>. Data spectra are represented by thick lines, and the averages of simulations by thin black lines. For the simulations, we show the 16 % and 84 % quantiles of the distribution with the same colours. This figure is taken from {{PlanckPapers|planck2016-l03}} (see detailled explanations there).
 
 
==Expected systematics and tests (bottom-up approach)==
 
 
 
Like all experiments, Planck-HFI had a number of specific issues that it needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather them together to give brief summaries of the issues and refer the reader to the appropriate section for more details.
 
 
 
* Cosmic rays &ndash; unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in [[TOI_processing#Glitch_statistics|the section on glitch statistics]]<!-- and in [[#Cosmic_rays|the section on cosmic rays]],--> as well as in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI glitch removal paper}}.
 
* "Elephants" &ndash; cosmic rays also hit the HFI 100-mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed with the rest of the thermal fluctuations, described directly below.
 
* Thermal fluctuations &ndash; HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in [[TOI_processing#Thermal_template_for_decorrelation|the timeline processing section on thermal decorrelation]]<!-- and in [[#1.6K_and_4K_stage_fluctuations|the section on 1.6-K and 4-K thermal fluctuations]]-->.
 
* Random telegraphic signal (RTS) or "popcorn noise" &ndash; some channels were occasionally affected by what seems to be a baseline that abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in [[TOI_processing#Noise_stationarity|the section on noise stationarity]]<!-- and [[#RTS_noise|the section on Random Telegraphic Signal Noise]]-->.
 
* Jumps &ndash; similar to (but distinct from) popcorn noise, small jumps were occasionally found in the data streams. These jumps are usually corrected, as described in [[TOI_processing#jump_correction|the section on jump corrections]].
 
* 4-K cooler-induced EM noise &ndash; the 4-K cooler induced noise in the detectors with very specific frequency signatures, which can be filtered. This is described in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}<!--, [[#4K_lines_Residuals|the section below on 4-K line residuals]]-->; their stability is discussed in [[TOI_processing#4K_cooler_lines_variability|the section on 4-K cooler line stability]].
 
* Compression &ndash; on-board compression is used to overcome our telemetry bandwidth limitations. This is explained in {{PlanckPapers|planck2011-1-5}}.
 
* Noise correlations &ndash; correlations in noise between detectors seems to be negligible, except for two polarization-sensitive detectors in the same horn. This is discussed in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Glitch removal paper}}.
 
* Pointing &ndash; the final pointing reconstruction for Planck is near the arcsecond level. This is discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}.
 
* Focal plane geometry &ndash; the relative positions of different horns in the focal plane are reconstructed using planets. This is also discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC paper}}.
 
* Main beam &ndash; the main beams for HFI are discussed in the {{PlanckPapers|planck2013-p03d|1|2013 Beams and Transfer function paper}}.
 
* Ruze envelope &ndash; random imperfections, or dust on the mirrors, can mildly increase the size of the beam. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
 
* Dimpling &ndash; the mirror support structure causes a pattern of small imperfections in the beams, which generate small sidelobe responses outside the main beam. This is discussed in the {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
 
* Far sidelobes &ndash; small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflections off the baffles. While small, when the Galactic centre is in the right position, this can be detected in the highest frequency channels, and so is removed from the data. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}} and also in {{PlanckPapers|planck2013-pip88|1|the 2013 Zodiacal emission paper}}.
 
* Planet fluxes &ndash; comparing the known flux densities of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimetre channels. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Mapmaking and Calibration paper}}.
 
* Point source fluxes &ndash; as with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Mapmaking and Calibration paper}}.
 
* Time constants &ndash; the HFI bolometers do not react instantaneously to light; there are small time constants, discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
 
* ADC correction &ndash; the HFI analogue-to-digital converters are not perfect, and are not used perfectly. Their effects on the calibration are discussed in {{PlanckPapers|planck2013-p03c|1|the 2013 Mapmaking and Calibration paper}}.
 
<!--* Gain changes with temperature changes-->
 
<!--* Optical cross-talk &ndash; this is negligible, as noted in [[#Optical_Cross-Talk|the optical cross-talk note]]. -->
 
* Bandpass &ndash; the transmission curves, or "bandpasses" have shown up in a number of places. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 spectral response paper}}.
 
<!--* Saturation &ndash; while this is mostly an issue only for Jupiter observations, it should be remembered that the HFI detectors cannot observe arbitrarily bright objects. This is discussed in [[#Saturation|the section below on saturation]].-->
 
 
 
<!---==Generic approach to systematics==
 
 
 
<font style="color:red;font-size:300%">This section is Under (Re-)Construction</font>
 
  
Some (null) tests done at the map level are described in {{PlanckPapers|planck2013-p03b}} and {{PlanckPapers|planck2014-a09}}.
+
<center>
 +
[[File:newpte.pdf|500px]]
 +
</center>
  
Some further tests will be described in the "CMB Power spectra and liklihood paper".
 
  
Finally detailed end-to-end simulations (from lowest level instrument behaviour to maps) are still ongoing for a detailed characterization, which will accompany the 100-217GHz polarisation maps when they are made available, probably by the summer of 2015.--->
 
  
 
==References==
 
==References==

Latest revision as of 08:40, 3 July 2018


The overall internal validation of the frequency maps is performed thanks to several tests:

  • difference between the PR2 (2015) and PR3 (2018) frequency maps,
  • survey difference maps for the PR2 and the PR3 frequency maps,
  • spectra of the PR2 and the PR3 data splits,
  • comparison of the FFP10 simulations and the PR3 data.


Frequency maps for the PR2 and the PR3 and their difference

This table shows the PR2 and PR3 maps and their differences in I, Q, and U. This table is complementary of the figure in Planck-2020-A3[1] (see detailled explanations there).

Comparaison of PR2 and PR3 I, Q and U maps and their difference.
PR2 frequency maps PR3 frequency maps difference
I Q U I Q U I Q U
100 GHz 100GHz DX11 I.pdf.pdf 100GHz DX11 Q.pdf.pdf 100GHz DX11 U.pdf.pdf 100GHz I.pdf 100GHz Q.pdf 100GHz U.pdf 100GHz diff I.pdf.pdf 100GHz diff Q.pdf.pdf 100GHz diff U.pdf.pdf
143 GHz 143GHz DX11 I.pdf.pdf 143GHz DX11 Q.pdf.pdf 143GHz DX11 U.pdf.pdf 143GHz I.pdf 143GHz Q.pdf 143GHz U.pdf 143GHz diff I.pdf.pdf 143GHz diff Q.pdf.pdf 143GHz diff U.pdf.pdf
217 GHz 217GHz DX11 I.pdf.pdf 217GHz DX11 Q.pdf.pdf 217GHz DX11 U.pdf.pdf 217GHz I.pdf 217GHz Q.pdf 217GHz U.pdf 217GHz diff I.pdf.pdf 217GHz diff Q.pdf.pdf 217GHz diff U.pdf.pdf
353 GHz 353GHz DX11 I.pdf.pdf 353GHz DX11 Q.pdf.pdf 353GHz DX11 U.pdf.pdf 353GHz I.pdf 353GHz Q.pdf 353GHz U.pdf 353GHz diff I.pdf.pdf 353GHz diff Q.pdf.pdf 353GHz diff U.pdf.pdf
545 GHz 545GHz DX11 I.pdf.pdf . . 545GHz I.pdf . . 545GHz diff I.pdf.pdf . .
857 GHz 857GHz DX11 I.pdf.pdf . . 857GHz I.pdf . . 857GHz diff I.pdf.pdf . .



Survey difference maps for the PR2 and the PR3 data

This table shows the PR2 and PR3 survey difference maps ((S1+S3)-(S2+S4))in I, Q, and U. This table is taken from Planck-2020-A3[1] (see detailled explanations there).

Comparaison of PR2 and PR3 I, Q and U survey difference maps.
PR2 survey difference maps PR3 survey difference maps
I Q U I Q U
100 GHz 100GHz DX11 surveyS1S3 S2S4 I.pdf 100GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 100GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 100GHz RD12RC4 surveyS1S3 S2S4 I.pdf 100GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 100GHz RD12RC4 surveyS1S3 S2S4 U.pdf
143 GHz 143GHz DX11 surveyS1S3 S2S4 I.pdf 143GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 143GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 143GHz RD12RC4 surveyS1S3 S2S4 I.pdf 143GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 143GHz RD12RC4 surveyS1S3 S2S4 U.pdf
217 GHz 217GHz DX11 surveyS1S3 S2S4 I.pdf 217GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 217GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 217GHz RD12RC4 surveyS1S3 S2S4 I.pdf 217GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 217GHz RD12RC4 surveyS1S3 S2S4 U.pdf
353 GHz 353GHz DX11 surveyS1S3 S2S4 I.pdf 353GHz DX11 surveyS1S3 S2S4 Q.pdf.pdf 353GHz DX11 surveyS1S3 S2S4 U.pdf.pdf 353GHz RD12RC4 surveyS1S3 S2S4 I.pdf 353GHz RD12RC4 surveyS1S3 S2S4 Q.pdf 353GHz RD12RC4 surveyS1S3 S2S4 U.pdf
545 GHz 545GHz DX11 surveyS1S3 S2S4 I.pdf . . 545GHz RD12RC4 surveyS1S3 S2S4 I.pdf . .
857 GHz 857GHz DX11 surveyS1S3 S2S4 I.pdf . . 857GHz RD12RC4 surveyS1S3 S2S4 I.pdf . .



Spectra of the PR2 and the PR3 data splits

This figure shows the EE and BB spectra of the PR2 and PR3 detset, half-mission and rings (for PR3 only) maps at 100, 143, 217, and 353 GHz. The auto-spectra of the difference maps and the cross-spectra between the maps are shown. The sky fraction used here is 43 %. The bins are: bin=1 for [math]2\leq\ell\lt 30[/math]; bin=5 for [math]30\leq\ell\lt 50[/math]; bin=10 for [math]50\leq\ell\lt 160[/math]; bin=20 for [math]160\leq\ell\lt 1000[/math]; and bin=100 for [math]\ell\gt 1000[/math]. This figure is taken from Planck-2020-A3[1] (see detailled explanations there).

Cl fsky43 DX11 RD12RC4 3000 oddeven multiplot.pdf



Comparison of the FFP10 simulated noise and systematic residuals and the PR3 data

This figure shows the noise and systematic residuals in TT, EE, BB, and EB spectra, at the three CMB frequencies, for difference maps of the ring (red) and half-mission (blue) null tests binned by [math]\Delta \ell =10[/math]. Data spectra are represented by thick lines, and the averages of simulations by thin black lines. For the simulations, we show the 16 % and 84 % quantiles of the distribution with the same colours. This figure is taken from Planck-2020-A3[1] (see detailled explanations there).

Newpte.pdf


References[edit]

Cosmic Microwave background