Difference between revisions of "The HFI DPC"
m (→HFI DPC Infrastructures) |
m |
||
| Line 3: | Line 3: | ||
| − | == Level 1: building the reference database during operations == | + | == Level 1: building the reference database during flight operations == |
(L1): consists in receiving the telemetry and ancillary data files and ingesting them into the DPC database. This involves decompressing, in some cases changing data formats, computing the time of individual data samples from the time of the compression slices, but otherwise no processing proper. Other steps are: | (L1): consists in receiving the telemetry and ancillary data files and ingesting them into the DPC database. This involves decompressing, in some cases changing data formats, computing the time of individual data samples from the time of the compression slices, but otherwise no processing proper. Other steps are: | ||
| Line 13: | Line 13: | ||
This is further described in the [[Pre-processing]] section | This is further described in the [[Pre-processing]] section | ||
| − | == Level 2: converting temporal information into maps == | + | == Level 2: converting temporal information into clean calibrated maps == |
(L2): this is where the data are processed from timelines into maps. The main processing steps are | (L2): this is where the data are processed from timelines into maps. The main processing steps are | ||
| Line 23: | Line 23: | ||
* The resulting data characteristics are given in section [[Summary]] | * The resulting data characteristics are given in section [[Summary]] | ||
| − | == Level 3: Frequency maps | + | == Level 3: Basic analyses of L2 Frequency maps== |
| − | (L3): This is where the data in the form of frequency maps are converted to catalogues and astrophysical component maps. Much of this is done in common with the LFI, and is further described in the [[HL | HFI/LFI common sections ]] | + | (L3): This is where the data in the form of frequency maps are converted to catalogues and full sky astrophysical component maps. Much of this is done in common with the LFI DPC, and is further described in the [[HL | HFI/LFI common sections ]] |
== Level 4 : Delivering results == | == Level 4 : Delivering results == | ||
| Line 31: | Line 31: | ||
Level 4 is the "Archive Level". No processing done, but rather exporting, reformating, documenting. | Level 4 is the "Archive Level". No processing done, but rather exporting, reformating, documenting. | ||
| − | == Level S : A common simulation software == | + | == Level S : A common HFI/LFI simulation software == |
| − | Level S is the so-called "Simulation Level", common to both instruments, and further described in the HFI/LFI [[ HL | common section]]. HFI specific developments are described in [[HFI-Validation | the HFI data validation section]]. | + | Level S is the so-called "Simulation Level", common to both instruments, and further described in the HFI/LFI [[ HL | common section]]. HFI specific developments (configuration control & MC bench, specific effects like 4K lines, glitches, ADC non-linearity, etc.) are described in [[HFI-Validation | the HFI data validation section]]. |
== HFI DPC Infrastructures== | == HFI DPC Infrastructures== | ||
Revision as of 17:06, 20 October 2012
The HFI DPC has been organized into different "Levels": 1, 2, 3, 4 and "S". In brief, during operations, L1 feeds the database resulting in time-ordered information (TOI) objects. L2 is the core of the processing, which turns TOIs into clean calibrated sky maps. L3 transforms these maps at specific frequencies into more scientific products, like catalogues, maps and spectra of astrophysical components. L3 can rely on simulation provided by the LS, while L4 refers to delivering the DPC products to ESA. This processing relies on dedicated software and hardware infrastructures developed pre-launch.
Contents
Level 1: building the reference database during flight operations[edit]
(L1): consists in receiving the telemetry and ancillary data files and ingesting them into the DPC database. This involves decompressing, in some cases changing data formats, computing the time of individual data samples from the time of the compression slices, but otherwise no processing proper. Other steps are:
- data ingestion (science, HK, ancillary, other?)
- construction of ToS in science data group
- pointing interpolation
- construction of other TOI and ROI objects from AHF, ...
This is further described in the Pre-processing section
Level 2: converting temporal information into clean calibrated maps[edit]
(L2): this is where the data are processed from timelines into maps. The main processing steps are
- Timeline (or Time-Ordered Information = TOI) processing, which includes conversion from ADUs to engineering units (Volts), demodulation, deglitching, conversion from engineering to physical units (Watts), removal of known systematic effects (non-linearities, 4K lines, Jumps, ring flagging), removal of the instrumental signature (time transfer function), temporal noise Estimation. Details here.
- Pointing and beam of each detector. See section Pointing&Beams.
- map-making & photomoetric calibration: projecting the TOIs onto all-sky maps, etc. See section Map-making.
- Characterisation/validation through angular power spectra. See section PowerSpectra.
- Overal HFI data validation, through difference tests, comparison to detailed simulations, etc., See section HFI-Validation
- The resulting data characteristics are given in section Summary
Level 3: Basic analyses of L2 Frequency maps[edit]
(L3): This is where the data in the form of frequency maps are converted to catalogues and full sky astrophysical component maps. Much of this is done in common with the LFI DPC, and is further described in the HFI/LFI common sections
Level 4 : Delivering results[edit]
Level 4 is the "Archive Level". No processing done, but rather exporting, reformating, documenting.
Level S : A common HFI/LFI simulation software[edit]
Level S is the so-called "Simulation Level", common to both instruments, and further described in the HFI/LFI common section. HFI specific developments (configuration control & MC bench, specific effects like 4K lines, glitches, ADC non-linearity, etc.) are described in the HFI data validation section.
HFI DPC Infrastructures[edit]
The HFI Data Processing Centre can be thought of as a centralized backbone providing hardware and software infrastructures to a relatively large number of geographically distributed groups of developers and other R&D groups in the HFI and LFI core teams
Further details were given in the HFI data processing paper of Planck early results series. In particular:
- Code and configuration management can be found at Planck early paper VI .
- Data management at Planck early paper VI .
- Instrument model (IMO) database at Planck early paper VI .
- Data flow management at Planck early paper VI .
- Hardware at Planck early paper VI .
(Planck) High Frequency Instrument
Data Processing Center
European Space Agency
House Keeping
Time of Sample
Ring-Ordered Information (DMC group/object)
Attitude History File
EMI/EMC influence of the 4K cooler mechanical motion on the bolometer readout electronics.
(Planck) Low Frequency Instrument
analog to digital converter
