Difference between revisions of "HFI-Validation"

From Planck Legacy Archive Wiki
Jump to: navigation, search
(→‎Generic approach to systematics)
Line 1: Line 1:
−
{{DISPLAYTITLE:Internal overall validation}}
+
{{DISPLAYTITLE:Overall internal validation}}
  
−
The HFI validation is mostly modular. That is, each part of the pipeline, be it timeline processing, map-making, or any other, validates the results of its work at each step of the processing, looking specifically for known issues. In addition, we do additional validation with an eye towards overall system integrity by looking at generic differences between sets of maps, in which most problems will become apparent, whether known or not. Both these are described below.  
+
HFI validation is mostly modular. In other words, each part of the pipeline, e.g., timeline processing or mapmaking, has the results of its work validated at each step of the processing, looking specifically for known issues. In addition, we perform additional validation with an eye towards overall system integrity, by looking at generic differences between sets of maps, in which most problems will become apparent (whether known or not). Both these approaches are described below.  
  
 
==Expected systematics and tests (bottom-up approach)==
 
==Expected systematics and tests (bottom-up approach)==
  
−
Like all experiments, Planck/HFI had a number of "issues" which it needed to track and verify were not compromising the data. While these are discussed in appropriate sections, here we gather them together to give brief summaries of the issues and refer the reader to the appropriate section for more details.  
+
Like all experiments, Planck-HFI had a number of specific issues that it needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather them together to give brief summaries of the issues and refer the reader to the appropriate section for more details.  
  
−
* Cosmic Rays - Unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than previous experiments. These were detected, the worst parts of the data flagged as unusable, and "tails" were modeled and removed. This is described in [[TOI_processing#Glitch_statistics|the section on glitch statistics]]<!-- and in [[#Cosmic_rays|the section on cosmic rays]],--> as well as in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Cosmic Ray Removal paper}}.
+
* Cosmic rays - unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in [[TOI_processing#Glitch_statistics|the section on glitch statistics]]<!-- and in [[#Cosmic_rays|the section on cosmic rays]],--> as well as in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI glitch removal paper}}.
−
* Elephants - Cosmic rays also hit the HFI 100 mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These are removed with the rest of the thermal fluctuations, described directly below.  
+
* "Elephants" - cosmic rays also hit the HFI 100 mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed with the rest of the thermal fluctuations, described directly below.  
−
* Thermal Fluctuations - HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in [[TOI_processing#Thermal_template_for_decorrelation|the timeline processing section on thermal decorrelation]]<!-- and in [[#1.6K_and_4K_stage_fluctuations|the section on 1.6 K and 4 K thermal fluctuations]]-->.
+
* Thermal fluctuations - HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in [[TOI_processing#Thermal_template_for_decorrelation|the timeline processing section on thermal decorrelation]]<!-- and in [[#1.6K_and_4K_stage_fluctuations|the section on 1.6-K and 4-K thermal fluctuations]]-->.
−
* Random Telegraphic Signal (RTS) or Popcorn Noise - Some channels were occasionally affected by what seems to be a baseline which abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in [[TOI_processing#Noise_stationarity|the section on noise stationarity]]<!-- and [[#RTS_noise|the section on Random Telegraphic Signal Noise]]-->.
+
* Random telegraphic signal (RTS) or "popcorn noise" - some channels were occasionally affected by what seems to be a baseline that abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in [[TOI_processing#Noise_stationarity|the section on noise stationarity]]<!-- and [[#RTS_noise|the section on Random Telegraphic Signal Noise]]-->.
−
* Jumps - Similar to but distinct from popcorn noise, small jumps were occasionally found in the data streams. These data are usually corrected. This is described in [[TOI_processing#jump_correction|the section on jump corrections]].  
+
* Jumps - similar to but distinct from popcorn noise, small jumps were occasionally found in the data streams. These data are usually corrected. This is described in [[TOI_processing#jump_correction|the section on jump corrections]].  
−
* 4 K Cooler-Induced EM Noise - The 4 K cooler induced noise in the detectors with very specific frequency signatures, which is filtered. This is described in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}<!--, [[#4K_lines_Residuals|the section below on 4 K line residuals]]-->, and their stability is discussed in [[TOI_processing#4K_cooler_lines_variability|the section on 4 K cooler line stability]].
+
* 4-K cooler-Induced EM Noise - the 4 K cooler induced noise in the detectors with very specific frequency signatures, which is filtered. This is described in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}<!--, [[#4K_lines_Residuals|the section below on 4 K line residuals]]-->, and their stability is discussed in [[TOI_processing#4K_cooler_lines_variability|the section on 4 K cooler line stability]].
−
* Compression - On-board compression is used to overcome our telemetry bandwidth limitations. This is explained in {{PlanckPapers|planck2011-1-5}}.  
+
* Compression - on-board compression is used to overcome our telemetry bandwidth limitations. This is explained in {{PlanckPapers|planck2011-1-5}}.  
−
* Noise Correlations - Correlations in noise between detectors seems to be negligible but for two polarization sensitive detectors in the same horn. This is discussed in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Cosmic Ray Removal paper}}.
+
* Noise correlations - Correlations in noise between detectors seems to be negligible but for two polarization sensitive detectors in the same horn. This is discussed in {{PlanckPapers|planck2013-p03e|1|the 2013 HFI Cosmic Ray Removal paper}}.
−
* Pointing - The final pointing reconstruction for Planck is near the arcsecond level. This is discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}.
+
* Pointing - the final pointing reconstruction for Planck is near the arcsecond level. This is discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC Paper}}.
−
* Focal Plane Geometry - The relative positions of different horns in the focal plane is reconstructed using planets. This is also discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC paper}}.  
+
* Focal plane geometry - the relative positions of different horns in the focal plane is reconstructed using planets. This is also discussed in {{PlanckPapers|planck2013-p03|1|the 2013 HFI DPC paper}}.  
−
* Main Beam - The main beams for HFI are discussed in the {{PlanckPapers|planck2013-p03d|1|2013 Beams and Transfer function paper}}.  
+
* Main beam - the main beams for HFI are discussed in the {{PlanckPapers|planck2013-p03d|1|2013 Beams and Transfer function paper}}.  
−
* Ruze Envelope - Random imperfections or dust on the mirrors can increase the size of the beam a bit. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
+
* Ruze envelope - random imperfections or dust on the mirrors can increase the size of the beam a bit. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
−
* Dimpling - The mirror support structure causes a pattern of small imperfections in the beams, which cause small sidelobe responses outside the main beam. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
+
* Dimpling - the mirror support structure causes a pattern of small imperfections in the beams, which cause small sidelobe responses outside the main beam. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.
−
* Far Sidelobes - Small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflecting off the baffles. While small, when the Galactic center is in the right position, this can be detected in the highest frequency channels, so this is removed from the data. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}} and, non-intuitively, {{PlanckPapers|planck2013-pip88|1|the 2013 Zodiacal emission paper}}.  
+
* Far sidelobes - small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflecting off the baffles. While small, when the Galactic center is in the right position, this can be detected in the highest frequency channels, so this is removed from the data. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}} and, non-intuitively, {{PlanckPapers|planck2013-pip88|1|the 2013 Zodiacal emission paper}}.  
−
* Planet Fluxes - Comparing the known fluxes of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimeter channels. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Map-Making and Calibration Paper}}.  
+
* Planet fluxes - comparing the known fluxes of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimeter channels. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Map-Making and Calibration Paper}}.  
−
* Point Source Fluxes - As with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Map-Making and Calibration paper}}.  
+
* Point source fluxes - as with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in {{PlanckPapers|planck2013-p03b|1|the 2013 Map-Making and Calibration paper}}.  
−
* Time Constants - The HFI bolometers do not react instantaneously to light; there are small time constants, discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.  
+
* Time constants - the HFI bolometers do not react instantaneously to light; there are small time constants, discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 Beams and Transfer function paper}}.  
−
* ADC Correction - The HFI Analog-to-Digital Converters are not perfect, and are not used perfectly. While this is an on-going effort, their effects on the calibration are discussed in {{PlanckPapers|planck2013-p03c|1|the 2013 Map-Making and Calibration paper}}.
+
* ADC correction - the HFI Analog-to-Digital Converters are not perfect, and are not used perfectly. While this is an on-going effort, their effects on the calibration are discussed in {{PlanckPapers|planck2013-p03c|1|the 2013 Map-Making and Calibration paper}}.
−
<!--* Gain changes with Temperature Changes-->
+
<!--* Gain changes with temperature changes-->
−
<!--* Optical Cross-Talk - This is negligible, as noted in [[#Optical_Cross-Talk|the optical cross-talk note]]. -->
+
<!--* Optical cross-talk - this is negligible, as noted in [[#Optical_Cross-Talk|the optical cross-talk note]]. -->
−
* Bandpass - The transmission curves, or "bandpass" has shown up in a number of places. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 spectral response paper}}.  
+
* Bandpass - the transmission curves, or "bandpass" has shown up in a number of places. This is discussed in {{PlanckPapers|planck2013-p03d|1|the 2013 spectral response paper}}.  
−
<!--* Saturation - While this is mostly an issue only for Jupiter observations, it should be remembered that the HFI detectors cannot observe arbitrarily bright objects. This is discussed in [[#Saturation|the section below on saturation]].-->
+
<!--* Saturation - ehile this is mostly an issue only for Jupiter observations, it should be remembered that the HFI detectors cannot observe arbitrarily bright objects. This is discussed in [[#Saturation|the section below on saturation]].-->
  
 
<!---==Generic approach to systematics==
 
<!---==Generic approach to systematics==

Revision as of 07:27, 20 June 2015


HFI validation is mostly modular. In other words, each part of the pipeline, e.g., timeline processing or mapmaking, has the results of its work validated at each step of the processing, looking specifically for known issues. In addition, we perform additional validation with an eye towards overall system integrity, by looking at generic differences between sets of maps, in which most problems will become apparent (whether known or not). Both these approaches are described below.

Expected systematics and tests (bottom-up approach)[edit]

Like all experiments, Planck-HFI had a number of specific issues that it needed to be tracked to verify that they were not compromising the data. While these are discussed in appropriate sections, here we gather them together to give brief summaries of the issues and refer the reader to the appropriate section for more details.

  • Cosmic rays - unprotected by the atmosphere and more sensitive than previous bolometric experiments, HFI saw many more cosmic ray hits than its predecessors. These were detected, the worst parts of the data flagged as unusable, and "tails" were modelled and removed. This is described in the section on glitch statistics as well as in the 2013 HFI glitch removal paper[1].
  • "Elephants" - cosmic rays also hit the HFI 100 mK stage and cause the temperature to vary, inducing small temperature and thus noise variations in the detectors. These elephants are removed with the rest of the thermal fluctuations, described directly below.
  • Thermal fluctuations - HFI is an extremely stable instrument, but there are small thermal fluctuations. These are discussed in the timeline processing section on thermal decorrelation.
  • Random telegraphic signal (RTS) or "popcorn noise" - some channels were occasionally affected by what seems to be a baseline that abruptly changes between two levels, which has been variously called popcorn noise or random telegraphic signal. These data are usually flagged. This is described in the section on noise stationarity.
  • Jumps - similar to but distinct from popcorn noise, small jumps were occasionally found in the data streams. These data are usually corrected. This is described in the section on jump corrections.
  • 4-K cooler-Induced EM Noise - the 4 K cooler induced noise in the detectors with very specific frequency signatures, which is filtered. This is described in the 2013 HFI DPC Paper[2], and their stability is discussed in the section on 4 K cooler line stability.
  • Compression - on-board compression is used to overcome our telemetry bandwidth limitations. This is explained in Planck-Early-IV[3].
  • Noise correlations - Correlations in noise between detectors seems to be negligible but for two polarization sensitive detectors in the same horn. This is discussed in the 2013 HFI Cosmic Ray Removal paper[1].
  • Pointing - the final pointing reconstruction for Planck is near the arcsecond level. This is discussed in the 2013 HFI DPC Paper[2].
  • Focal plane geometry - the relative positions of different horns in the focal plane is reconstructed using planets. This is also discussed in the 2013 HFI DPC paper[2].
  • Main beam - the main beams for HFI are discussed in the 2013 Beams and Transfer function paper[4].
  • Ruze envelope - random imperfections or dust on the mirrors can increase the size of the beam a bit. This is discussed in the 2013 Beams and Transfer function paper[4].
  • Dimpling - the mirror support structure causes a pattern of small imperfections in the beams, which cause small sidelobe responses outside the main beam. This is discussed in the 2013 Beams and Transfer function paper[4].
  • Far sidelobes - small amounts of light can sometimes hit the detectors from just above the primary or secondary mirrors, or even from reflecting off the baffles. While small, when the Galactic center is in the right position, this can be detected in the highest frequency channels, so this is removed from the data. This is discussed in the 2013 Beams and Transfer function paper[4] and, non-intuitively, the 2013 Zodiacal emission paper[5].
  • Planet fluxes - comparing the known fluxes of planets with the calibration on the CMB dipole is a useful check of calibration for the CMB channels, and is the primary calibration source for the submillimeter channels. This is done in the 2013 Map-Making and Calibration Paper[6].
  • Point source fluxes - as with planet fluxes, we also compare fluxes of known, bright point sources with the CMB dipole calibration. This is done in the 2013 Map-Making and Calibration paper[6].
  • Time constants - the HFI bolometers do not react instantaneously to light; there are small time constants, discussed in the 2013 Beams and Transfer function paper[4].
  • ADC correction - the HFI Analog-to-Digital Converters are not perfect, and are not used perfectly. While this is an on-going effort, their effects on the calibration are discussed in the 2013 Map-Making and Calibration paper[7].
  • Bandpass - the transmission curves, or "bandpass" has shown up in a number of places. This is discussed in the 2013 spectral response paper[4].


References[edit]

  1. ↑ 1.01.1 Planck 2013 results. X. HFI energetic particle effects: characterization, removal, and simulation, Planck Collaboration, 2014, A&A, 571, A10.
  2. ↑ 2.02.12.2 Planck 2013 results. VI. High Frequency Instrument Data Processing, Planck Collaboration, 2014, A&A, 571, A6.
  3. ↑ Planck early results, IV. First assessment of the High Frequency Instrument in-flight performance, Planck HFI Core Team, A&A, 536, A4, (2011).
  4. ↑ 4.04.14.24.34.44.5 Planck 2013 results. IX. HFI spectral response, Planck Collaboration, 2014, A&A, 571, A9.
  5. ↑ Planck 2013 results. XIV. Zodiacal emission, Planck Collaboration, 2014, A&A, 571, A14.
  6. ↑ 6.06.1 Planck 2013 results. VIII. HFI photometric calibration and Map-making, Planck Collaboration, 2014, A&A, 571, A8.
  7. ↑ Planck 2013 results. VII. HFI time response and beams, Planck Collaboration, 2014, A&A, 571, A7.

(Planck) High Frequency Instrument

random telegraphic signal

Cosmic Microwave background

analog to digital converter

Retrieved from ‘https://wiki.cosmos.esa.int/planck-legacy-archive/index.php?title=HFI-Validation&oldid=11689’