Difference between revisions of "HFI-systematics"
m |
|||
| (4 intermediate revisions by 2 users not shown) | |||
| Line 3: | Line 3: | ||
Like all experiments, ''Planck'' HFI had a number of specific issues that needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather the systematic effects affecting the TOIs together to give brief summaries of the issues and refer the reader to the appropriate sections for more details. | Like all experiments, ''Planck'' HFI had a number of specific issues that needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather the systematic effects affecting the TOIs together to give brief summaries of the issues and refer the reader to the appropriate sections for more details. | ||
| − | * Cosmic rays – unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in [[TOI_processing#Glitch_statistics|the section on glitch statistics]]<!-- and in [[#Cosmic_rays|the section on cosmic rays]],--> as well as in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI glitch removal paper}}. An estimate of the level and effect of remaining undetected glitches is described in {{PlanckPapers|planck2016-l03}}. | + | * 4-K cooler-induced EM noise – the 4-K cooler induced noise in the detectors with very specific frequency signatures, which can be filtered. This is described in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}<!--, [[#4K_lines_Residuals|the section below on 4-K line residuals]]-->; their stability is discussed in [[TOI_processing#4K_cooler_lines_variability|the section on 4-K cooler line stability]]. |
| + | * ADC – {{PlanckPapers|planck2013-p03c}} reported that the HFI raw data show apparent gain variations with time of up to 2% due to nonlinearities in the HFI readout chain. In the 2013 data release ({{PlanckPapers|planck2013-p03b}}) a correction for this systematic error was applied as an apparent gain variation at the map making stage. The 2013 maps relied on an effective gain correction based on the consistency constraints from the reconstructed sky maps, which proved to be sufficient for the 2013 cosmological analysis based on temperature only. For the 2015 data release we have implemented a direct ADC correction in the TOI. The ADC effect and its correction, and its validation through end-to-end simulations are described in sections 2 and 5 of {{PlanckPapers||planck2014-a08}}. An improved correction for the ADC affect using the SRoll map-making algorithm was implemented for the 2018 release, and is described in {{PlanckPapers|planck2016-XLVI}} and {{PlanckPapers|planck2016-l03}}. | ||
| + | * Compression – on-board compression is used to overcome our telemetry bandwidth limitations. This is explained in {{PlanckPapers|planck2011-1-5}}and an analysis of this effect in maps is described in {{PlanckPapers|planck2016-l03}}. | ||
| + | * Cosmic rays – unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in [[TOI_processing#Glitch_statistics|the section on glitch statistics]]<!-- and in [[#Cosmic_rays|the section on cosmic rays]],--> as well as in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI glitch removal paper}}. An estimate of the level and effect of remaining undetected glitches is described in {{PlanckPapers|planck2016-l03}}. This shows a very likely mechanism to account for the 1/f detector noises which present the same knee frequencies independently of different noise levels. | ||
| + | * Cross-talk – The signal cross-talk between different housings is negligible. The cross-talk between two bolometers in the same housing was detected (through coincident glitches). The effect of this thermal cross-talk has been estimated in {{PlanckPapers|planck2016-l03}}. | ||
| + | * Dimpling – the mirror support structure causes a pattern of small imperfections in the beams, which generate small sidelobe responses outside the main beam. This is discussed in the {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}. | ||
* "Elephants" – cosmic rays also hit the HFI 100-mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed from the timelines with the rest of the thermal fluctuations, described directly below. | * "Elephants" – cosmic rays also hit the HFI 100-mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed from the timelines with the rest of the thermal fluctuations, described directly below. | ||
| − | * | + | * Far sidelobes – small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflections off the baffles. While small, when the Galactic centre is in the right position, this can be detected in the highest frequency channels, and so is removed from the data. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}} and also in {{PlanckPapers|planck2013-pip88|1|the 2013 Zodiacal emission paper}} for 2013, in {{PlanckPapers|planck2014-a09}} for 2015 and in {{PlanckPapers|planck2016-l03}} for 2018. |
| − | |||
* Jumps – similar to (but distinct from) popcorn noise, small jumps were occasionally found in the data streams. These jumps are usually corrected, as described in [[TOI_processing#jump_correction|the section on jump corrections]]. | * Jumps – similar to (but distinct from) popcorn noise, small jumps were occasionally found in the data streams. These jumps are usually corrected, as described in [[TOI_processing#jump_correction|the section on jump corrections]]. | ||
| − | * | + | * Main beam – the main beams for HFI are discussed in the {{PlanckPapers|planck2013-p03d|1|2013 Beams and Transfer function paper}} and an improved analysis for the 2015 release is presented in {{PlanckPapers|planck2014-a08}}. |
| − | |||
| − | |||
* Noise correlations – correlations in noise between detectors seems to be negligible, except for two polarization-sensitive detectors in the same horn. This is discussed in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Glitch removal paper}}. | * Noise correlations – correlations in noise between detectors seems to be negligible, except for two polarization-sensitive detectors in the same horn. This is discussed in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Glitch removal paper}}. | ||
| + | * Planet fluxes – comparing the known flux densities of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimetre channels. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Mapmaking and Calibration paper}}. In {{PlanckPapers|planck2016-l03}} the planet-based calibration at 545 and 857 GHz is compared to the dipole-based calibration. The best estimate of planet fluxes is presented in {{PlanckPapers|planck2017-LII}}. | ||
| + | * Point source fluxes – as with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Mapmaking and Calibration paper}}. | ||
* Pointing and Focal Plane geometry – the final pointing reconstruction for Planck is near the arcsecond level, as discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}} and in {{PlanckPapers|planck2013-p01}}. The relative positions of different horns in the focal plane are reconstructed using planets. This is also discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC paper}}. An improved reconstruction of the focal plane directions was used for the 2015 and 2018 releases and is described in {{PlanckPapers|planck2014-a01}}. | * Pointing and Focal Plane geometry – the final pointing reconstruction for Planck is near the arcsecond level, as discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}} and in {{PlanckPapers|planck2013-p01}}. The relative positions of different horns in the focal plane are reconstructed using planets. This is also discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC paper}}. An improved reconstruction of the focal plane directions was used for the 2015 and 2018 releases and is described in {{PlanckPapers|planck2014-a01}}. | ||
| − | * | + | * Polarization efficiencies – relative polarization efficiencies of bolometers within a frequency band, with respect to the average, kept at the same value of PR2, have been extracted from the redundancy of the data. The effect on the maps has been estimated by simulation in {{PlanckPapers|planck2016-l03}}. |
| + | * Random telegraphic signal (RTS) or "popcorn noise" – some channels were occasionally affected by what seems to be a baseline that abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in [[TOI_processing#Noise_stationarity|the section on noise stationarity]]<!-- and [[#RTS_noise|the section on Random Telegraphic Signal Noise]]-->. | ||
* Ruze envelope – random imperfections, or dust on the mirrors, can mildly increase the size of the beam. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}. | * Ruze envelope – random imperfections, or dust on the mirrors, can mildly increase the size of the beam. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}. | ||
| − | * | + | * Thermal fluctuations – HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in [[TOI_processing#Thermal_template_for_decorrelation|the timeline processing section on thermal decorrelation]]<!-- and in [[#1.6K_and_4K_stage_fluctuations|the section on 1.6-K and 4-K thermal fluctuations]]-->. |
| − | * | + | * Very long constants – Time constants up to 2s have been discussed and removed in the PR1 and PR2 TOI processing. (PR3 TOI processing is the same as the PR2 one). Longer time constants by an order of magnitude have been detected by the phase shift they introduced on the dipole. This effect was taken into account in the PR2 release. Taking advantage of the redundancy in the data, the Very Long Time constants (real and imaginary parts) have been solved for as a function of the frequency harmonics, but real part was detected and corrected for only at 353 GHz. Unfortunately this introduced residual artifacts in the form of stripping, not aligned with the scanning strategy (called « zebra pattern »), which was demonstrated by simulations. These artifacts are visible in the odd-even survey map null tests {{PlanckPapers|planck2016-l03}}. |
| − | |||
| − | |||
| − | The systematic effects affecting the maps were corrected in open loop in the 2015 release, using ground measurements and modelisation. For the 2018 release, almost all known systematics affecting the maps have been corrected. Using together ground-based and extracted-from-the-sky determination of the parameters of these systematic effects, we correct these at the mapmaking level using the [[Map-making|''SRoll'']] generalized polarized | + | The systematic effects affecting the maps were corrected in open loop in the 2015 release, using ground measurements and modelisation. For the 2018 release, almost all known systematics affecting the maps have been corrected. Using together ground-based and extracted-from-the-sky determination of the parameters of these systematic effects, we correct these at the mapmaking level using the [[Map-making|''SRoll'']] generalized polarized destriper. |
| − | + | A full description of the remaining systematic effects is included in {{PlanckPapers|planck2016-l03}}. The two figures below extracted from that paper provide a snapshot of the levels of the main effects remaining in the 2018 polarization maps, estimated using realistic simulations and/or jack-knife tests on the data itself. | |
| − | |||
| − | |||
| − | |||
| + | [[File:Screen Shot 2018-07-03 at 09.59.26.png|thumb|500px|center]] | ||
| − | + | [[File:Screen Shot 2018-07-03 at 10.00.18.png|thumb|500px|center]] | |
| − | + | In 2015, maps were dominated by systematic effects at low multipoles, which were filtered out in the PR2 delivered maps. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | In {{PlanckPapers|planck2016-XLVI}}, the Sroll mapmaking was introduced an reduced the residuals, allowing the first measurement of the polarized signal at very low multipoles. | |
| − | |||
| + | For PR3 frequency maps, in polarization, for the CMB channels (100 to 217 GHz), all systematic effects are negligible except the ADC non linearity correction which is close to the noise at very low multipoles. At 353 GHz, the transfer function, the bandpass mismatch leakage, and the polarization efficiency all exceeds the noise at very low multipoles ({{PlanckPapers|planck2016-l03}}). | ||
==References== | ==References== | ||
<References /> | <References /> | ||
Latest revision as of 07:38, 18 January 2019
Like all experiments, Planck HFI had a number of specific issues that needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather the systematic effects affecting the TOIs together to give brief summaries of the issues and refer the reader to the appropriate sections for more details.
- 4-K cooler-induced EM noise – the 4-K cooler induced noise in the detectors with very specific frequency signatures, which can be filtered. This is described in the 2013 HFI DPC Paper[1]; their stability is discussed in the section on 4-K cooler line stability.
- ADC – Planck-2013-VII[2] reported that the HFI raw data show apparent gain variations with time of up to 2% due to nonlinearities in the HFI readout chain. In the 2013 data release (Planck-2013-VIII[3]) a correction for this systematic error was applied as an apparent gain variation at the map making stage. The 2013 maps relied on an effective gain correction based on the consistency constraints from the reconstructed sky maps, which proved to be sufficient for the 2013 cosmological analysis based on temperature only. For the 2015 data release we have implemented a direct ADC correction in the TOI. The ADC effect and its correction, and its validation through end-to-end simulations are described in sections 2 and 5 of [4]. An improved correction for the ADC affect using the SRoll map-making algorithm was implemented for the 2018 release, and is described in Planck-2016-XLVI[5] and Planck-2020-A3[6].
- Compression – on-board compression is used to overcome our telemetry bandwidth limitations. This is explained in Planck-Early-IV[7]and an analysis of this effect in maps is described in Planck-2020-A3[6].
- Cosmic rays – unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in the section on glitch statistics as well as in the 2013 HFI glitch removal paper[8]. An estimate of the level and effect of remaining undetected glitches is described in Planck-2020-A3[6]. This shows a very likely mechanism to account for the 1/f detector noises which present the same knee frequencies independently of different noise levels.
- Cross-talk – The signal cross-talk between different housings is negligible. The cross-talk between two bolometers in the same housing was detected (through coincident glitches). The effect of this thermal cross-talk has been estimated in Planck-2020-A3[6].
- Dimpling – the mirror support structure causes a pattern of small imperfections in the beams, which generate small sidelobe responses outside the main beam. This is discussed in the the 2013 Beams and Transfer function paper[9].
- "Elephants" – cosmic rays also hit the HFI 100-mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed from the timelines with the rest of the thermal fluctuations, described directly below.
- Far sidelobes – small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflections off the baffles. While small, when the Galactic centre is in the right position, this can be detected in the highest frequency channels, and so is removed from the data. This is discussed in the 2013 Beams and Transfer function paper[9] and also in the 2013 Zodiacal emission paper[10] for 2013, in Planck-2015-A08[11] for 2015 and in Planck-2020-A3[6] for 2018.
- Jumps – similar to (but distinct from) popcorn noise, small jumps were occasionally found in the data streams. These jumps are usually corrected, as described in the section on jump corrections.
- Main beam – the main beams for HFI are discussed in the 2013 Beams and Transfer function paper[9] and an improved analysis for the 2015 release is presented in Planck-2015-A07[12].
- Noise correlations – correlations in noise between detectors seems to be negligible, except for two polarization-sensitive detectors in the same horn. This is discussed in the 2013 HFI Glitch removal paper[8].
- Planet fluxes – comparing the known flux densities of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimetre channels. This is done in the 2013 Mapmaking and Calibration paper[3]. In Planck-2020-A3[6] the planet-based calibration at 545 and 857 GHz is compared to the dipole-based calibration. The best estimate of planet fluxes is presented in Planck-2017-LII[13].
- Point source fluxes – as with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in the 2013 Mapmaking and Calibration paper[3].
- Pointing and Focal Plane geometry – the final pointing reconstruction for Planck is near the arcsecond level, as discussed in the 2013 HFI DPC Paper[1] and in Planck-2013-I[14]. The relative positions of different horns in the focal plane are reconstructed using planets. This is also discussed in the 2013 HFI DPC paper[1]. An improved reconstruction of the focal plane directions was used for the 2015 and 2018 releases and is described in Planck-2015-A01[15].
- Polarization efficiencies – relative polarization efficiencies of bolometers within a frequency band, with respect to the average, kept at the same value of PR2, have been extracted from the redundancy of the data. The effect on the maps has been estimated by simulation in Planck-2020-A3[6].
- Random telegraphic signal (RTS) or "popcorn noise" – some channels were occasionally affected by what seems to be a baseline that abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in the section on noise stationarity.
- Ruze envelope – random imperfections, or dust on the mirrors, can mildly increase the size of the beam. This is discussed in the 2013 Beams and Transfer function paper[9].
- Thermal fluctuations – HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in the timeline processing section on thermal decorrelation.
- Very long constants – Time constants up to 2s have been discussed and removed in the PR1 and PR2 TOI processing. (PR3 TOI processing is the same as the PR2 one). Longer time constants by an order of magnitude have been detected by the phase shift they introduced on the dipole. This effect was taken into account in the PR2 release. Taking advantage of the redundancy in the data, the Very Long Time constants (real and imaginary parts) have been solved for as a function of the frequency harmonics, but real part was detected and corrected for only at 353 GHz. Unfortunately this introduced residual artifacts in the form of stripping, not aligned with the scanning strategy (called « zebra pattern »), which was demonstrated by simulations. These artifacts are visible in the odd-even survey map null tests Planck-2020-A3[6].
The systematic effects affecting the maps were corrected in open loop in the 2015 release, using ground measurements and modelisation. For the 2018 release, almost all known systematics affecting the maps have been corrected. Using together ground-based and extracted-from-the-sky determination of the parameters of these systematic effects, we correct these at the mapmaking level using the SRoll generalized polarized destriper.
A full description of the remaining systematic effects is included in Planck-2020-A3[6]. The two figures below extracted from that paper provide a snapshot of the levels of the main effects remaining in the 2018 polarization maps, estimated using realistic simulations and/or jack-knife tests on the data itself.
In 2015, maps were dominated by systematic effects at low multipoles, which were filtered out in the PR2 delivered maps.
In Planck-2016-XLVI[5], the Sroll mapmaking was introduced an reduced the residuals, allowing the first measurement of the polarized signal at very low multipoles.
For PR3 frequency maps, in polarization, for the CMB channels (100 to 217 GHz), all systematic effects are negligible except the ADC non linearity correction which is close to the noise at very low multipoles. At 353 GHz, the transfer function, the bandpass mismatch leakage, and the polarization efficiency all exceeds the noise at very low multipoles (Planck-2020-A3[6]).
References[edit]
- ↑ 1.01.11.2 Planck 2013 results. VI. High Frequency Instrument Data Processing, Planck Collaboration, 2014, A&A, 571, A6.
- ↑ Planck 2013 results. VII. HFI time response and beams, Planck Collaboration, 2014, A&A, 571, A7.
- ↑ 3.03.13.2 Planck 2013 results. VIII. HFI photometric calibration and Map-making, Planck Collaboration, 2014, A&A, 571, A8.
- ↑
- ↑ 5.05.1 Planck intermediate results. XLVI. Reduction of large-scale systematic effects in HFI polarization maps and estimation of the reionization optical depth, Planck Collaboration Int. XLVI A&A, 596, A107, (2016).
- ↑ 6.06.16.26.36.46.56.66.76.86.9 Planck 2018 results. III. High Frequency Instrument data processing and frequency maps, Planck Collaboration, 2020, A&A, 641, A3.
- ↑ Planck early results, IV. First assessment of the High Frequency Instrument in-flight performance, Planck HFI Core Team, A&A, 536, A4, (2011).
- ↑ 8.08.1 Planck 2013 results. X. HFI energetic particle effects: characterization, removal, and simulation, Planck Collaboration, 2014, A&A, 571, A10.
- ↑ 9.09.19.29.3 Planck 2013 results. IX. HFI spectral response, Planck Collaboration, 2014, A&A, 571, A9.
- ↑ Planck 2013 results. XIV. Zodiacal emission, Planck Collaboration, 2014, A&A, 571, A14.
- ↑ Planck 2015 results. VIII. High Frequency Instrument data processing: Calibration and maps, Planck Collaboration, 2016, A&A, 594, A8.
- ↑ Planck 2015 results. VII. High Frequency Instrument data processing: Time-ordered information and beam processing, Planck Collaboration, 2016, A&A, 594, A7.
- ↑ Planck intermediate results. LII. Planets flux densities, Planck Collaboration Int. LII A&A, 607, A122, (2017)
- ↑ Planck 2013 results. I. Overview of Products and Results, Planck Collaboration, 2014, A&A, 571, A1.
- ↑ Planck 2015 results. I. Overview of products and results, Planck Collaboration, 2016, A&A, 594, A1.
(Planck) High Frequency Instrument
analog to digital converter
Cosmic Microwave background
random telegraphic signal
