Difference between revisions of "Timelines"
(→HFI processing) |
|||
(68 intermediate revisions by 9 users not shown) | |||
Line 1: | Line 1: | ||
+ | {{DISPLAYTITLE: Timelines}} | ||
==General description== | ==General description== | ||
− | The timelines, or TOIs for | + | The timelines, or TOIs for "time-ordered information," are vectors of signal or of pointing or of some other quantity, giving the value of that quantity as a function of time during the mission. For LFI, refer to {{PlanckPapers|planck2014-a03}} and {{PlanckPapers|planck2016-l02}}, and for HFI, to {{PlanckPapers|planck2014-a08}}. |
− | + | The PLA contains two kinds of TOIs: | |
− | * | + | * "Semi-raw" TOIs consists of minimally processed data (in Volt units) |
− | * | + | * "Calibrated" TOIs consists of data which has been cleaned of artifacts and systematic effects, and converted to physical units. |
− | The | + | The Semi-raw timelines are applicable to all data releases. The Calibrated timelines of HFI are applicable to both 2015 and 2017 releases; note that for the 2017 release, further cleaning and calibration steps are carried out by the map-making algorithm SRoll, which are not reflected in the calibrated timelines ({{PlanckPapers|planck2016-l03}}). |
− | The | + | The TOIs described here are sampled regularly at the (instrument-dependent) detector sampling frequency and span the full science mission. They thus consist of a large amount of data. This section is concerned with the signal TOIs and their associated pointing TOIs. For selected detectors the DPCs provide a single timeline of cleaned and calibrated signal and three timelines of coordinates, corresponding to the two angular coordinates and one orientation angle. |
− | < | + | Obviously these vectors are very long (about 1.38×10<sup>10</sup> samples for HFI, from 2.5×10<sup>6</sup> to 5.5×10<sup>6</sup> for LFI) and thus need to be split into multiple files for export. Here the data are split by operational day (OD) as follows: |
+ | * HFI has 12 files per OD, one for pointing and one for signal for each channel, covering ODs 91 to 974, all sampled at <i>F</i><sub>samp</sub> = 180.3737 Hz; | ||
+ | * LFI has six files per OD, three each for signal and pointing, since the sampling frequency is channel dependent, and covering ODs 91 to 1543, sampled at <i>F</i><sub>samp</sub> = 32.5079, <i>F</i><sub>samp</sub> = 46.5455, and <i>F</i><sub>samp</sub> = 78.7692 Hz. | ||
− | + | The signal timelines are encoded as single-precision real values, but the pointing vectors had to be encoded as double-precision reals to maintain the required accuracy. The result is that the total volume of the full dataset is approximately 30 TB. All files also contain the OBT. | |
− | < | + | The signal timelines have been cleaned of all known instrumental systematic effects, they have been calibrated in flux, and the solar and Earth motion dipole signals have been removed. But they have not been cleaned of the low-frequency, "1/<i>f</i>" noise that needs to be removed via a destriping tool. The methods consists of removing offsets or "baselines" determined by minimizing the differences in the signals at points on the sky where they intersect. For HFI an offset per ring is determined; for LFI the the baseline is computed every 0.246, 0.988, and 1.000 seconds for the 30, 44, and 70 GHz channels, respectively and maintain the same structure of the signal timelines. |
− | + | In the case of HFI these offsets are determined using the full mission and all the valid detectors per channel, and they are then used for all the maps produced, i.e., for those maps using any fraction of the mission (year, survey) or any subset of the detectors (single detector, detector set), and it has been shown that using offsets determined from a limited part of the mission and/or a subset of the detectors yields maps that are less consistent than otherwise. | |
− | + | In the case of LFI these offsets are determined using the full mission and all the valid detectors per channel; those values have been used for the production of the full mission period maps. Note that baseline used for shorter period maps are determined on those data periods to avoid noise cross-correlation effects and those are not delivered. | |
− | + | The offsets are delivered separately, as described below. | |
− | + | Furthermore, for HFI and LFI, there are three sets of offsets produced: the primary set using the full rings; and the secondary ones using the first and second half rings, only. The differences between the primary and secondary sets are fairly minor, but they are necessary to include to reconstruct the maps as they were built by the HFI-DPC and LFI-DPC. In particular the offsets are useful for users wishing to build a map of a small area of the sky. | |
− | HFI timelines at the DPC are indexed from 0 to | + | The HFI delivers its offsets in a ROI, or "ring-ordered Information" file. That file contains a table of <i>N</i><sub>rings</sub> = 26766 rows by <i>N</i><sub>bolometers</sub> in which each cell contains a 3-element vector containing the full-ring, first half-ring, and second half-ring offsets. The indices of the signal TOI delimiting the rings are given in a second ROI file with the same <i>N</i><sub>rings</sub> rows containing global parameters. |
+ | |||
+ | The LFI delivers its offsets in a TOI format, the structure is <i>exactly</i> the same as that used by the science timelines. The user can simply subtract one-by-one the offset timelines from the science timelines and then generate a map using the result. | ||
+ | In the case of the half-ring baselines, a vector has been added in the OBT extension; this vector contains "1" or "2," depending which half ring it should be applied to. | ||
+ | |||
+ | ===HFI indexing=== | ||
+ | |||
+ | Each FITS file contains keywords giving the first and last index of the section of the full TOI contained in the file. These indices are those of the internal DPC representation; they begin at 0, corresponding to the switch-on of the instrument, and run to some very large number (about 25 billion for HFI) representing the full mission. Only the <i>science</i> part of the mission is exported, which is about 60% of the total for HFI. Left out are the early mission phase (cool-down and validation phases) and the warm phase of HFI, during which LFI continued to collect data. The HFI keywords also indicate which rings are included in each OD, but note that normally an OD does not begin at the same time (with the same index) as a ring. | ||
+ | |||
+ | Note that for historical reasons, the OD definitions of the two DPCs differ; for LFI they occur at the transition between pointing periods, whereas for HFI they do not. However, this has no significance for the user, since this splitting is somewhat arbitrary anyway, and what counts is the full vector once it is rebuilt in the user's own work space. | ||
+ | |||
+ | HFI timelines at the DPC are indexed from 0 to 25×10<sup>9</sup>, which correspond to instrument switch-on and switch-off, respectively. Of these the indices 1.4×10<sup>9</sup> to 151.5×10<sup>9</sup> correspond to the science mission and are exported and delivered. Each file contains a keyword giving the first and last index of the data in that file, and EndIndex(OD)+1 = BeginIndex (OD+1). The ROI "Global" file gives the Begin/End Indices of each Ring, or Pointing Period, and can be used to destripe the signal TOIs with the offsets provided in the "Destriping-Offsets" ROI file. | ||
==Production process== | ==Production process== | ||
Line 31: | Line 45: | ||
=== HFI processing === | === HFI processing === | ||
− | The input TOIs are in ADUs representing the voltage signal at the end of the electronics (see [[HFI_detection_chain | Detection chain]] for details). The processing applied to remove instrumental effects and to calibrate them is described in detail in the [[TOI processing | HFI TOI processing]] section, and in | + | ==== Calibrated timelines ==== |
+ | |||
+ | The input TOIs are in ADUs, representing the voltage signal at the end of the electronics (see [[HFI_detection_chain | Detection chain]] for details). The processing applied to remove instrumental effects and to calibrate them is described in detail in the [[TOI processing | HFI TOI processing]] section, and in {{PlanckPapers|planck2014-a08}} we give a very brief summary here for convenience. This pipeline performs the following operations. | ||
+ | |||
+ | ; ADC correction: This corrects for the uneven size of the ADC bins (see [[ADC correction]]). | ||
+ | ; Demodulation: This is performed around a variable level, which is determined from the valid input data (using a validity flag from a previous version of the processing), and the data are converted to engineering units (V) using known conversion coefficients. | ||
+ | ; Despiking: The redundancy within a ring is used to determine where glitches occur. Once identified, the glitches are fitted with templates. A glitch flag is produced that identifies the strongest part of the glitches, and a timeline of glitch tails. produced from the template fits, is subtracted from the demodulated timeline. Finally, the flagged regions are replaced with data from an average over the pointing period in order to not leave holes in the timeline that would perturb the usage of Fourier methods on them | ||
+ | ; Dark template removal: The two dark bolometers are demodulated and despiked as above; the resulting timelines are then smoothed and used as an indicator of the overall temperature variations of the bolometer plate. Where the variations are consistent with each other, they are combined and removed from the bolometer signal timelines using appropriate coupling coefficients. The few percent of the data where they are not consistent are flagged as not valid on the timelines. | ||
+ | ; Conversion to absorbed power: The timeline is converted to watts of absorbed power using the bolometer function. This includes a nonlinearity correction and removal of the 4-K cooler lines (i.e., the electromagnetic interference of the 4-K cooler with the bolometer readout wires, which induces some sharp lines in the signal power spectra at frequencies of the 4-K cooler's fundamental and its multiples, folded by the signal modulations). Fourier coefficients of the relevant lines are determined on a per-ring basis, and then removed from the data. The quality of the removal depends on the bolometer. | ||
+ | ; Deconvolution by the time transfer function: This is done to correct for the non-instantaneous time response of the bolometers. The function itself is modeled using up to eight time constants, which are adjusted primarily on planet data and also from comparisons of the northward and southward scans of the Galactic Plane. It is then removed using Fourier techniques. | ||
+ | ; Jump correction: This corrects some rare jumps in the signal baseline (there are on average around 0.3 jumps per bolometer per pointing period). The jumps are detected and characterized on smoothed TOIs, and corrected by adding a constant to part of the signal timeline. The origin of the jumps is not known. | ||
− | + | The results of this set of processing steps are a timeline of signal (in absorbed watts) and a "valid data" flag timeline for each of the 50 valid bolometers that are processed, out of the 52 HFI bolometers; these timelines contain the full-sky signal, i.e., including the solar and orbital dipoles, the Zodiacal light, and contributions from the far sidelobes. The dipoles are necessary for the flux calibration and are removed at the mapmaking stage. The remaining two bolometers (143-8 and 545-3) show semi-random jumps in the signal level, typically jumping over 2 to 5 different "pseudo-baseline" levels, a behaviour known as "random telegraphic signal" (RTS), so that these are commonly called the RTS bolometers. Finally, ring-level statistics of different types (mean, median, rms, kurtosis, etc.) are determined on a per-ring basis for all timelines, and a selection based on these statistics is used to discard anomalous rings, which are recorded in a ring-level flag for each valid bolometer timeline (see the [[TOI_processing#Discarded_rings | Discarded rings]] section). | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | Throughout this processing, some planet signals are masked in the timeline in order to avoid ringing effects in the processing. This concerns only Jupiter at 100 and 143 GHz, Jupiter and Saturn at 217 GHz, and Jupiter, Saturn, and Mars at 353, 545, and 857 GHz (thanks to various improvements in the despiking and in the transfer functions it is no longer necessary to flag these three planets in the CMB channels, as was done in the 2013 release). Since they move on the sky, the portion of the sky masked during one survey is observed during another one, and no hole is left in the final map. In parallel, the planet data are processed in a similar way, but with different parameters for the despiking step. These results are processed separately to determine the beam shapes and the focal plane geometry. | |
− | + | Next, the TOIs are calibrated in astrophysical units using the results of the calibration pipeline (see the [[Map-making | Map-making and calibration]] section) and the best estimates of the zero-point offsets (a constant level for each bolometer). These values are given in the RIMO. Also, the solar and Earth-motion dipole signals are computed and removed. | |
− | + | These TOIs are accompanied by several flags that are described below. The most important one is the "Total" flag, which identifies all the samples that were discarded by the DPC mapmaking, as described in [[TOI processing | HFI TOI processing]]. This flag includes the portion beyond 72 min for the rings that are longer; this is not used for science analysis because of the slight drift of the satellite pointing direction (spin axis) during these long acquisition periods. | |
− | + | At this point the TOIs still contain the low frequency (1/<i>f</i>) noise, which should be removed before projection onto a map. That cleaning step is called "destriping" or "baseline removal". The HFI-DPC does its destriping at ring-level, meaning that a constant is added to the signal of each ring in order to minimize the difference at the ring crossings, where the signal should be the same for all detectors (with the necessary precautions as described in [[Map-making | Map-making and calibration]] section); should the user want to use the DPC's offsets, they are provided separately (see [[Timelines#ROI_files | ROI files]] below). Maps produced from these TOIs, and after subtraction of the DPC's destriping offsets, are not identical to the maps that are delivered. This is discussed in Section A.2 of {{PlanckPapers|planck2014-a09}}. | |
− | + | As indicated above, the brightest planets are masked in the TOI in order to avoid ringing problems. For users specifically wanting to study these planets, we provide separate timelines covering just the planet transits. These timelines also include transits of Uranus and Neptune, which are not masked in the regular TOIs. They are produced with less agressive deglitching options in order to work on the rapidly changing baselines. These are the data used the reconstruction of the focal plane geometry and also for the determination of the scanning beams. | |
+ | |||
+ | The table below lists the 16 transits, giving for each the begin/end ring number and the begin/end OD in which that ring is found. Note that transit 15 is split into two parts because some tests were done during the transit. The last row is a region processed in the same manner but without a planet transit, which is included for comparison. | ||
+ | |||
+ | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=700px | ||
+ | |+ '''Planet transits''' | ||
+ | |- bgcolor="ffdead" | ||
+ | ! Number || Beg Ring || End Ring || Beg OD || End OD || Comment | ||
+ | |- | ||
+ | | 1 || 2700 || 3211 || 160 || 176 || Jupiter, Mars, Neptune | ||
+ | |- | ||
+ | | 2 || 3892 || 4099 || 203 || 212 || Uranus | ||
+ | |- | ||
+ | | 3 || 4575 || 4775 || 233 || 240 || Saturn | ||
+ | |- | ||
+ | | 4 || 7900 || 8150 || 330 || 337 || Mars | ||
+ | |- | ||
+ | | 5 || 8979 || 9188 || 367 || 376 || Neptune | ||
+ | |- | ||
+ | | 6 || 9550 || 9750 || 392 || 402 || Saturn | ||
+ | |- | ||
+ | | 7 || 9940 || 10300 || 411 || 425 || Jupiter, Uranus | ||
+ | |- | ||
+ | | 8 || 14018 || 14227 || 537 || 544 || Neptune | ||
+ | |- | ||
+ | | 9 || 14820 || 15190 || 567 || 583 || Jupiter, Uranus | ||
+ | |- | ||
+ | | 10 || 15925 || 16275 || 612 || 624 || Saturn | ||
+ | |- | ||
+ | | 11 || 20016 || 20225 || 734 || 743 || Neptune | ||
+ | |- | ||
+ | | 12 || 20900 || 21650 || 775 || 804 || Saturn, Uranus | ||
+ | |- | ||
+ | | 13 || 21780 || 22150 || 808 || 818 || Jupiter | ||
+ | |- | ||
+ | | 14 || 24992 || 25202 || 916 || 921 || Neptune | ||
+ | |- | ||
+ | | 15a || 25830 || 25849 || 937 || 938 || Mars, Uranus | ||
+ | |- | ||
+ | | 15b || 25864 || 26225 || 947 || 957 || Mars, Uranus | ||
+ | |- | ||
+ | | 16 || 26650 || 27005 || 968 || 974 || Jupiter | ||
+ | |- | ||
+ | | 17 || 12000 || 12150 || 479 || 483 || background | ||
+ | |} | ||
+ | |||
+ | |||
+ | The pointing (see common [[Detector_pointing | Detector Pointing]] section for details) is determined from the AHF produced by the MOC, which gives the direction and orientation of the LOS of a fiducial position in the focal plane at frequencies of 8Hz during stable pointing and 4Hz during manoeuvres. This is interpolated to the timestamps of data samples, corrected for the wobble and other time-dependent offsets determined from the observed positions of a large number of sources around the sky, and finally converted to the LOS of each detector using the quaternions in the IMO. These pointing data are valid for both the regular and the planet TOIs. | ||
+ | |||
+ | ==== Semi-raw timelines ==== | ||
+ | |||
+ | The only step which has been applied to the HFI semi-raw TOIs is the ADC non-linearity correction (to first order), which includes some level of 4k-line removal. The semi-raw timelines are in units of Volts and have not been de-modulated. | ||
+ | |||
+ | === LFI processing === | ||
+ | |||
+ | ==== Calibrated timelines ==== | ||
+ | |||
+ | The input TOIs are in ADUs, representing the voltage signal at the output of the electronics (see [[LFI design, qualification, and performance#Radiometer Chain Assembly, RCA, | Radiometer Chain Assembly (RCA)]] for details). The processing applied to remove instrumental effects and to calibrate them is described in detail in the [[TOI processing | LFI TOI processing]] section, in {{PlanckPapers|planck2014-a03||Planck-2015-A03}} and in {{PlanckPapers|planck2016-l02}}; here we give a very brief summary for convenience. That pipeline performs the following operations. | ||
+ | ; ADC correction: Due to ADC nonlinearity under certain condition, this instrumental effect is removed by applying well know templates directly to the diode signal. | ||
+ | ; Electronics spikes: This is caused by the interaction between the electronics clock and the scientific data lines. The signal is detected in all the LFI radiometer time-domain outputs as a 1-s square wave with a rising edge near 0.5s and a falling edge near 0.75s, synchronous with the onboard time signal. In the frequency domain it appears as a spike signal at multiples of 1Hz. The 44-GHz channels are the only LFI outputs significantly affected by this effect. Consequently the spike signal is removed from the data only in these channels. | ||
+ | ; Demodulation: Each LFI diode switches at 4096Hz between the sky and the 4-K reference load. The data acquired in this way are dominated by 1/<i>f</i> noise that is highly correlated between the two streams; differencing those streams results in a strong reduction of 1/<i>f</i> noise. The procedure applied is described in [[TOI processing | LFI TOI processing]], taking into account that the gain modulation factor <i>R</i> was computed on timestreams with the Galaxy and point sources masked to avoid strong sky signals. | ||
+ | ; Diode combination: Two detector diodes provide the output for each LFI receiver channel. To minimize the impact of imperfect isolation of the data stream of each diode, we perform a weighted average of the time-ordered data from the two diodes of each receiver. The procedure applied is detailed in [[TOI processing | LFI TOI processing]]; and the weights used are kept fixed for the entire mission. | ||
+ | ; Scientific calibration: This step calibrates the timelines to physical units K<sub>CMB</sub>, fitting the total CMB dipole convolved with the 4π beam representation, without taking into account the signature due to Galactic stray light. | ||
+ | ; Gain regularization: The calibration constants computed using the model of the dipole signal suffer from large uncertainties when the Planck spacecraft is badly aligned with the dipole axis. To reduce the noise, we apply an adaptive smoothing algorithm, which is also designed to preserve the discontinuities caused by abrupt changes in the working configuration of the radiometers (e.g., sudden temperature changes in the focal plane). | ||
+ | ; Removal of solar and orbital dipole signal: The combined solar and orbital dipole is convolved with the 4π beam representation of each radiometer and then removed from its timeline. | ||
+ | ; Removal of Galactic stray light: The light incident on the focal plane without reflecting on the primary mirror (stray light) is a major source of systematic effects, especially when the Galactic plane intersects the direction of the main spillover. This effect is corrected by removing the estimated straylight signal from the timelines. This signal is computed as the convolution of Galaxy simulation with the beam sidelobes (see details in [[TOI processing | LFI TOI processing]]). | ||
+ | |||
+ | After these processing steps the timelines are used for the production of the maps. | ||
+ | |||
+ | The pointing (see common [[Detector_pointing | Detector Pointing]] section for details) is determined from the AHF produced by the MOC, which gives the direction and orientation of the LOS of a fiducial position in the focal plane at frequencies of 8Hz during stable pointing and 4Hz during manoeuvres. This is interpolated to the timestamps of data samples, corrected for the wobble and other time-dependent offsets determined from the observed positions of a large number of sources around the sky, as well as temperature sensors, and finally converted to the LOS of each detector. | ||
+ | |||
+ | ==== Semi-raw timelines ==== | ||
+ | |||
+ | The steps which have been applied to the LFI semi-raw TOIs are: ADC correction, spike removal, and demodulation with R-factor application. The semi-raw timelines are in units of ADUs. | ||
− | |||
==File Names== | ==File Names== | ||
− | The file names are of the form | + | The file names are of the form |
− | ''{H,L}FI_TOI_{fff}-{SCI,PTG} | + | ''{H,L}FI_TOI_{fff}-{SCI,RAW,PTG,TAI,OFF}_Rn.nn_ODxxxx.fits'', |
where | where | ||
− | * ''fff'' denotes the frequency | + | * ''fff'' denotes the frequency, |
− | * SCI | + | * SCI denotes calibrated signal TOIs, |
− | * | + | * RAW denotes semi-raw signal TOIs, |
+ | * PTG denotes pointing TOIs, | ||
+ | * TAI denotes the OBT-MJD correlation TOI, | ||
+ | * OFF denotes the baseline TOI, | ||
+ | * Rn.nn is the version, and | ||
* ODxxxx indicates the OD. | * ODxxxx indicates the OD. | ||
+ | Regarding the HFI TOIs for the 100 to 217GHz channels, the R2.00, made public in Jan 2015, contained only the unpolarized bolometer timelines, while R2.02, made public in July 2015, contains all bolometers. | ||
− | + | The HFI planet timelines are named | |
+ | |||
+ | ''{H,L}FI_TOI_{fff}-SCI-planets_R2.nn_ODxxxx.fits'' | ||
+ | |||
+ | and are provided for all 50 valid bolometers, i.e., SWBs and PSBs. | ||
==FITS file structure== | ==FITS file structure== | ||
− | All FITS files begin with a primary extension containing no data and a minimal header, which is followed by one or more ''BINTABLE'' extensions | + | All FITS files begin with a primary extension containing no data and a minimal header, which is followed by one or more ''BINTABLE'' extensions containing data, and with a description of the data in the header keywords. In what follows we will usually ignore the primary extension, and count only the extensions containing data. |
===TOI files=== | ===TOI files=== | ||
− | The signal FITS files contain N , | + | The signal FITS files contain <i>N+1</i> , "BINTABLE", data extensions, where <i>N</i> is the number of detectors in that frequency channel. The first extension contains the OBT and its flags, called the "global" flags since they apply to all detectors, which is followed by one extension for each detector, containing the signal and its "local" flag. The flags columns are written as "byte" in which each of the 8 bits (maximum) encodes one flag timeline. The meaning of each bit is given in the header comments and is subject to change. For the global flag they include the following list. |
− | * | + | |
− | * | + | * For HFI: |
− | * | + | ** unstable pointing, where 1 = pointing is not stable (e.g., during repointing manoeuvres); |
− | * HCM | + | ** dark correlation, where 1 = darks are uncorrelated and data are flagged; |
− | * and more | + | ** first/Second half ring, which samples are in which half (only covers the stable pointing part of the ring); |
+ | ** HCM, whether in HCM mode (unstable pointing); | ||
+ | ** and more (see extension header for details). | ||
+ | |||
+ | * For LFI: | ||
+ | ** bit 0, unstable pointing, where 1 = pointing is not stable; | ||
+ | ** bit 1, time correlation quality, where 1 = outside specification; | ||
+ | ** bit 2, special observation, where 1 = special observation, such as a deep scan. | ||
+ | |||
+ | For the local flag they include the following list. | ||
− | + | * For HFI: | |
+ | ** total flag, being a combination of the various flags that is the one finally used in the mapmaking (all samples with "total flag" different from zero should not be used); | ||
+ | ** data not valid, e.g., glitched samples; | ||
+ | ** despike common (for PSBs only), corresponding to a glitch on the current or the other of a PSB pair; | ||
+ | ** strong signal, i.e., on the Galactic plane; | ||
+ | ** strong source, i.e., on a point source; | ||
+ | ** and more (see extension header for details). | ||
− | * | + | * For LFI: |
− | * | + | ** bit 0, data not valid, where 1 = science sample not valid; |
− | * | + | ** bit 2: planet crossing, where 1 = science sample containing planet; |
− | * | + | ** bit 3: moving objects, where 1 = minor Solar System object (not yet used); |
− | * | + | ** bit 4: gap, where 1 = this sample was artificially included due to a gap in the data. |
− | |||
+ | For HFI the header extension gives more details on the flags and the meaning of "1" and "0". | ||
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | ||
Line 95: | Line 209: | ||
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | !colspan=" | + | !colspan="5" | 1. EXTNAME = "OBT" : Data columns |
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | ! Column | + | ! Column name || Data type || Units || Description || Comment |
|- | |- | ||
− | |OBT || | + | |OBT || Double || 2<sup>-16</sup> sec || Onboard time || |
|- | |- | ||
− | |FLAG || Byte || | + | |FLAG || Byte || None || Various bit-level flags || |
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | ! Keyword || Data Type || Value || Description | + | ! Keyword || Data Type || Value || Description || Comment |
+ | |- | ||
+ | |OD || Int || || OD covered (as in filename) || | ||
|- | |- | ||
− | | | + | |BEGIDX || Int || || First sample index of given OD || Only HFI |
|- | |- | ||
− | | | + | |ENDIDX || Int || || Last sample index of given OD || Only HFI |
|- | |- | ||
− | | | + | |BEGRING || Int || || First ring in given OD || Only HFI |
|- | |- | ||
− | | | + | |ENDRING || Int || || Last ring in given OD || Only HFI |
|- | |- | ||
− | | | + | |TIMEZERO || String || 1958-01-01z00:00 || Origin of OBT || Only HFI |
|- | |- | ||
− | |TIMEZERO || | + | |TIMEZERO || Float || 106744000000000. || Origin of OBT || Only LFI |
− | + | ||
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | !colspan=" | + | !colspan="5" | n. EXTNAME = ''DETNAME'' : Data columns |
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | ! Column | + | ! Column name || Data type || Units || Description || Comment |
|- | |- | ||
− | |SIGNAL || Real*4 || K<sub>cmb</sub> or MJy/ | + | |SIGNAL || Real*4 || K<sub>cmb</sub> or MJy.sr<sup>-1</sup> || Value of signal || Comment |
|- | |- | ||
− | |FLAG || Byte || | + | |FLAG || Byte || None || Various bit-level flags || |
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | ! Keyword || Data | + | ! Keyword || Data type || Value || Description || Comment |
|- | |- | ||
− | |UNIT || String || || Units of signal | + | |UNIT || String || || Units of signal || Only HFI, LFI always K<sub>cmb</sub> |
|- | |- | ||
− | |DESTRIPE || 1/0 || || | + | |DESTRIPE || 1/0 || || Whether timeline is destriped || Only HFI |
|- | |- | ||
− | |OD || Int || || OD covered (as in filename) | + | |OD || Int || || OD covered (as in filename) || |
|- | |- | ||
− | |BEGIDX || Int || || | + | |BEGIDX || Int || || First sample index of given OD || Only HFI |
|- | |- | ||
− | |ENDIDX || Int || || | + | |ENDIDX || Int || || Last sample index of given OD || Only HFI |
|- | |- | ||
− | |BEGRING || Int || || | + | |BEGRING || Int || || First ring in given OD || Only HFI |
|- | |- | ||
− | |ENDRING || Int || || | + | |ENDRING || Int || || Last ring in given OD || Only HFI |
|} | |} | ||
+ | The HFI Planet files have the same structure, but the local flags contain a single "Data not valid" flag. Also, the first and last ODs of each transit are usually not complete, since they contain only the rings that are included in the transit observations. | ||
− | The pointing files have a similar structure, except that ''DETNAME'' extensions contain | + | The pointing files have a similar structure, except that ''DETNAME'' extensions contain three columns of Real*8 variables with the φ, θ, and ψ Galactic spherical coordinates of each sample in radians, as shown in the table below. There is no local flag for the coordinates. |
{| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | ||
Line 156: | Line 273: | ||
|- bgcolor="ffdead" | |- bgcolor="ffdead" | ||
− | ! Column | + | ! Column name || Data type || Units || Description |
|- | |- | ||
− | |PHI || Real*8 || | + | |PHI || Real*8 || Radian || Longitude |
|- | |- | ||
− | |THETA || Real*8 || | + | |THETA || Real*8 || Radian || Colatitude |
|- | |- | ||
− | |PSI || Real*8 || | + | |PSI || Real*8 || Radian || Roll angle |
|} | |} | ||
+ | ===HFI ROI files=== | ||
+ | |||
+ | The files provided by HFI are | ||
+ | * ''HFI_ROI_GlobalParams_RelNum_full.fits'', | ||
+ | * ''HFI_ROI_DestripeOffsets_RelNum_full.fits'', | ||
+ | which are described below. | ||
+ | |||
+ | ;Global parameters: This ROI file contains a single "BINTABLE" extension with three columns that give, for each ring, the index number of the first sample, the ESA pointing ID of the ring, and the ring start time. All three are encoded as Int*8, and the time is given in units of 2<sup>-16</sup> sec from "TIMEZERO", 1958-01-01z00:00, as in the TOI files. Rings are numbered from 240 to 27005, as given in the "BEGRING" and "ENDRING" keywords. | ||
+ | |||
+ | ; Destriping offsets: This ROI file contains a single "BINTABLE" extension with 3 columns that give, for each ring, the offsets to subtract from each bolometer signal timeline or vector. There are three offsets available for each bolometer (full-ring, first half-ring, and second half-ring), for building the full-ring or the two half-ring maps, respectively. The offsets must be subtracted from Index(<i>N</i>) to Index(<i>N+1</i>)-1 of the corresponding signal timeline, where <i>N</i> is the ring number, and the indices are given in the "global" parameters file. The offsets are given in the same units as the signal vectors, that is K<sub>cmb</sub> for the 100- to 353-GHz channels, and MJy.sr<sup>-1</sup> for the 545- and 857-GHz channels. All values are encoded as Real*4. The rings for which the offsets are set to 0.000 are [[TOI_processing#Discarded_rings | "badrings" ]] which are rejected in the mapmaking process. These rings are also flagged by the "TotalFlag", as are the portions of rings longer than about 72.5 min, where the drift in the satellite's spin axis becomes important. | ||
+ | |||
+ | ===TAI TOI files=== | ||
+ | The TAI TOI files contain one extension with two columns, the first is the OBT value (exactly the same as reported in the SCI TOI), the second is the corresponding modified Julian day. Note that leap seconds were not added. | ||
− | === | + | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px |
+ | |+ '''Time TAI TOI file data structure''' | ||
− | The files provided by HFI are | + | |- bgcolor="ffdead" |
− | * '' | + | !colspan="5" | 1. EXTNAME = "OBT-MJD" : Data columns |
− | * '' | + | |- bgcolor="ffdead" |
− | which are | + | ! Column name || Data type || Units || Description || Comment |
+ | |- | ||
+ | |OBT || Int*8 || 2<sup>-16</sup> sec || Onboard time || | ||
+ | |- | ||
+ | |MJD || Real*8 || day || Modified Julian day || | ||
+ | |} | ||
+ | |||
+ | ===LFI OFF TOI files=== | ||
+ | The OFF (baseline) TOI files adopt the same file structure as the Science TOI files. Note that in case of OFF timelines related to the half-ring, an additional column is included in the OBT extension to define for each sample if it belongs to half-ring 1 or half-ring 2. | ||
+ | |||
+ | ===LFI HouseKeeping files=== | ||
+ | House keeping timelines are: | ||
+ | * LFI_TOI_DAE-FAST-HK_R2.nn_ODxxxx.fits; | ||
+ | * LFI_TOI_DAE-SlowConfiguration-HK_R2.nn_ODxxxx.fits; | ||
+ | * LFI_TOI_DAE-SlowCurrent-HK_R2.nn_ODxxxx.fits; | ||
+ | * LFI_TOI_DAE-SlowPhaseSwitch-HK_R2.nn_ODxxxx.fits; | ||
+ | * LFI_TOI_DAE-SlowVoltage-HK_R2.nn_ODxxxx.fits; | ||
+ | * LFI_TOI_REBA-HK_R2.nn_ODxxxx.fits; | ||
+ | * SCS_TOI_EssentialASW-HK_R2.nn_ODxxxx.fits; | ||
+ | * SCS_TOI_NonEssentialASW-HK_R2.nn_ODxxxx.fits. | ||
+ | |||
+ | Each file contains two extensions, the first is the OBT (values are sampled every 1 or 10 seconds), while the second contains a variable number of columns equivalent to twice the number of Housekeeping timelines stored. Each Housekeeping datum is accompanied by its flag (normally "0" means that the value was considered invalid or out of limits). The Housekeeping names are the ones defined in the LFI Instrument Operation Manual. | ||
+ | |||
+ | == Other Releases: 2020 NPIPE == | ||
+ | |||
+ | <div class="toccolours mw-collapsible mw-collapsed" style="background-color: #9ef542;width:80%"> | ||
+ | '''2020 Release NPIPE timelines''' | ||
+ | <div class="mw-collapsible-content"> | ||
+ | The NPIPE release includes pointing information, as well as three flavours of timelines for both Planck instruments. | ||
+ | |||
+ | === Pointing === | ||
+ | |||
+ | NPIPE satellite attitude files are interpolated to detector time-stamps. Since all HFI detectors are sampled at the same rate, four versions cover all Planck sampling rates: 30GHz; 44GHz; 70GHz; and HFI frequencies. Unlike earlier releases, NPIPE pointing files include the low-frequency pointing corrections (PTCOR), so only aberration corrections are needed to reconstruct detector pointing. | ||
+ | |||
+ | Individual detector pointing is also supplied to support PLA facilities that require it. Given the large volume of the individual detector pointing files, most users are better served by the satellite pointing files. | ||
+ | |||
+ | === Raw timelines === | ||
+ | |||
+ | The raw timelines contain the detector signal with minimal processing. For LFI this means four separate timelines for each radiometer assembly, consisting of a sky and a load stream for each of the two diodes. LFI preprocessing collapses them into a single timeline for each radiometer. | ||
+ | |||
+ | Raw HFI timelines were corrected with the initial ADCNL profiles solved from warm HFI data acquired during the LFI extended mission. They include the HFI dark bolometer timelines that serve a similar function as the LFI load signal. | ||
+ | |||
+ | === Preprocessed timelines === | ||
+ | |||
+ | Preprocessed timelines are approximately calibrated to CMB kelvins. They are jump-corrected and anomalous pointing periods are flagged. LFI timelines are collapsed to a single timeline per radiometer and ADCNL-corrected using DPC-provided ADCNL profiles. | ||
+ | |||
+ | HFI timelines are decorrelated using dark bolometer data. Cosmic-ray glitches are located, subtracted and flagged. Bolometric nonlinearity is corrected for using ground-measured nonlinearity models. | ||
+ | |||
+ | === Reprocessed timelines === | ||
+ | |||
+ | Reprocessed timelines have been bandpass-mismatch and far sidelobe-corrected. Gain fluctuations have been corrected for by fitting a step-wise gain-fluctuation model. For HFI, the second-order ADCNL was measured and corrected for in the 100-217GHz channels. All these template corrections are solved on full-mission, full-frequency data. Residual errors are highly correlated, so it is not advised to split the reprocessed timelines to make, e.g., detector-set or half-mission maps. | ||
+ | |||
+ | The timelines were destriped with the Madam destriper code to suppress 1/<i>f</i> noise. For better flexibility, the 1/<i>f</i> noise was solved using only a single detector, depolarized signal. Small-scale, high-frequency noise in the timelines is approximately independent of other detectors, although secondary mechanisms (co-incident glitches or errors in the polarization template) introduce weak correlations. | ||
+ | |||
+ | ===File Names=== | ||
+ | |||
+ | Most file names are of the form | ||
+ | |||
+ | ''{H,L}FI_TOI_{fff}-{RAW,preprocessed,reprocessed,PTG}_Rn.nn_ODxxxx.fits'', | ||
+ | |||
+ | where | ||
+ | * ''fff'' denotes the frequency, | ||
+ | * ''RAW'' denotes raw signal TOIs, | ||
+ | * ''preprocessed'' denotes preprocessed TOIs, | ||
+ | * ''reprocessed'' denotes reprocessed TOIs, | ||
+ | * ''PTG'' denotes detector pointing TOIs, | ||
+ | * ''Rn.nn'' is the version, and | ||
+ | * ''ODxxxx'' indicates the OD. | ||
+ | |||
+ | Satellite pointing files are of the form | ||
+ | |||
+ | ''satellite_pointing_{030,044,070,HFI}_Rnnn_ODxxxx.fits''. | ||
+ | |||
+ | ===FITS file structure=== | ||
+ | |||
+ | All FITS files begin with a primary extension containing no data and a minimal header, which is followed by one or more ''BINTABLE'' extensions containing data, and with a description of the data in the header keywords. In what follows we will usually ignore the primary extension, and count only the extensions containing data. | ||
+ | |||
+ | ====TOI files==== | ||
+ | |||
+ | The signal FITS files contain <i>N+1</i> , "BINTABLE", data extensions, where <i>N</i> is the number of detectors in that frequency channel. The first extension contains the OBT and its flags, called the "global" flags since they apply to all detectors, which is followed by one extension for each detector, containing the signal and its "local" flag. The flags columns are written as "byte" in which each of the 8 bits (maximum) encodes one flag timeline. The meaning of each bit is given in the header comments and is subject to change. For the global flag they are as follows. | ||
+ | |||
+ | * For HFI RAW files: | ||
+ | ** unwritten, 1 = timestamps NOT written; | ||
+ | ** unstable pointing, where 1 = pointing is not stable (e.g., during repointing manoeuvres). | ||
+ | |||
+ | * For LFI RAW files: | ||
+ | ** bit 0, unstable pointing, where 1 = pointing is not stable. | ||
+ | |||
+ | For the local flag they include the following list. | ||
+ | |||
+ | * For LFI and HFI RAW data: | ||
+ | ** bit 0, data not valid, where 1 = science sample not valid. | ||
+ | |||
+ | Other flag bits are defined and explained in the file header but not used in NPIPE processing. | ||
+ | |||
+ | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | ||
+ | |+ '''RAW TOI file data structure''' | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 1. EXTNAME = "OBT" : Data columns | ||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |OBT || 64-bit integer || 2<sup>-16</sup> sec || Onboard time || LFI | ||
+ | |- | ||
+ | |OBT || Double || TAI nanosec || Onboard time || HFI | ||
+ | |- | ||
+ | |FLAG || Byte || None || Various bit-level flags || | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Keyword || Data type || Value || Description || Comment | ||
+ | |- | ||
+ | |BEGIDX || Int || || First sample index in file || Only HFI | ||
+ | |- | ||
+ | |ENDIDX || Int || || Last sample index in file || Only HFI | ||
+ | |- | ||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | n. EXTNAME = ''DETNAME'' : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |SKY || 64-bit float || ADU || Value of decompressed sky signal || LFI | ||
+ | |- | ||
+ | |REF || 64-bit float || ADU || Value of decompressed reference load signal || LFI | ||
+ | |- | ||
+ | |SIGNAL || 32-bit integer || ADU || Value of ADCNL-corrected signal || HFI | ||
+ | |- | ||
+ | |FLAG || Byte || None || Various bit-level flags || | ||
+ | |} | ||
+ | |||
+ | |||
+ | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | ||
+ | |+ '''(P)reprocessed TOI file data structure''' | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 1. EXTNAME = "OBT" : Data columns | ||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |OBT || 64-bit integer || 2<sup>-16</sup> sec || Onboard time || LFI | ||
+ | |- | ||
+ | |OBT || Double || TAI nanosec || Onboard time || HFI | ||
+ | |- | ||
+ | |FLAG || Byte || None || Various bit-level flags || | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Keyword || Data Type || Value || Description || Comment | ||
+ | |- | ||
+ | |BEGIDX || Int || || First sample index in file || Only HFI | ||
+ | |- | ||
+ | |ENDIDX || Int || || Last sample index in file || Only HFI | ||
+ | |- | ||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | n. EXTNAME = ''DETNAME'' : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |SIGNAL || Double || K<sub>CMB</sub> || Value of (p)reprocessed signal || | ||
+ | |- | ||
+ | |FLAG || Byte || None || Various bit-level flags || | ||
+ | |} | ||
+ | |||
+ | |||
+ | {| border="1" cellpadding="3" cellspacing="0" align="center" style="text-align:left" width=800px | ||
+ | |+ '''Satellite pointing file data structure''' | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 1. EXTNAME = "OBT" : Data columns | ||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |OBT || 64-bit integer || 2<sup>-16</sup> sec || Onboard time || LFI | ||
+ | |- | ||
+ | |OBT || Double || TAI nanosec || Onboard time || HFI | ||
+ | |- | ||
+ | |FLAG || Byte || None || Unvalid pointing flag || | ||
+ | |||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 2. EXTNAME = "spin_phase" : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |PHASE || Double || Degrees || Spacecraft spin phase || | ||
+ | |- | ||
+ | |FLAG || Byte || None || Bad data flag || | ||
+ | |||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 3. EXTNAME = "attitude" : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |QUATERNION_X || Double || None || Spacecraft attitude quaternion X-component || | ||
+ | |- | ||
+ | |QUATERNION_Y || Double || None || Spacecraft attitude quaternion Y-component || | ||
+ | |- | ||
+ | |QUATERNION_Z || Double || None || Spacecraft attitude quaternion Z-component || | ||
+ | |- | ||
+ | |QUATERNION_S || Double || None || Spacecraft attitude quaternion S-component || | ||
+ | |- | ||
+ | |FLAG || Byte || None || Bad data flag || | ||
+ | |||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 4. EXTNAME = "velocity" : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |X_VEL || Double || km s<sup>-1</sup> || Spacecraft barycentric velocity X-component || | ||
+ | |- | ||
+ | |Y_VEL || Double || km s<sup>-1</sup> || Spacecraft barycentric velocity Y-component || | ||
+ | |- | ||
+ | |Z_VEL || Double || km s<sup>-1</sup> || Spacecraft barycentric velocity Z-component || | ||
+ | |- | ||
+ | |FLAG || Byte || None || Bad data flag || | ||
+ | |||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | !colspan="5" | 5. EXTNAME = "position" : Data columns | ||
+ | |||
+ | |- bgcolor="ffdead" | ||
+ | ! Column name || Data type || Units || Description || Comment | ||
+ | |- | ||
+ | |X_POS || Double || AU || Spacecraft barycentric position X-component || HFI only | ||
+ | |- | ||
+ | |Y_POS || Double || AU || Spacecraft barycentric position Y-component || HFI only | ||
+ | |- | ||
+ | |Z_POS || Double || AU || Spacecraft barycentric position Z-component || HFI only | ||
+ | |- | ||
+ | |FLAG || Byte || None || Bad data flag || | ||
+ | |||
+ | |||
+ | |} | ||
+ | |||
+ | |||
+ | </div> | ||
+ | </div> | ||
− | |||
− | + | == References == | |
− | < | + | <References /> |
[[Category:Mission products|001]] | [[Category:Mission products|001]] |
Latest revision as of 11:30, 22 June 2021
Contents
General description[edit]
The timelines, or TOIs for "time-ordered information," are vectors of signal or of pointing or of some other quantity, giving the value of that quantity as a function of time during the mission. For LFI, refer to Planck-2015-A02[1] and Planck-2020-A2[2], and for HFI, to Planck-2015-A07[3].
The PLA contains two kinds of TOIs:
- "Semi-raw" TOIs consists of minimally processed data (in Volt units)
- "Calibrated" TOIs consists of data which has been cleaned of artifacts and systematic effects, and converted to physical units.
The Semi-raw timelines are applicable to all data releases. The Calibrated timelines of HFI are applicable to both 2015 and 2017 releases; note that for the 2017 release, further cleaning and calibration steps are carried out by the map-making algorithm SRoll, which are not reflected in the calibrated timelines (Planck-2020-A3[4]).
The TOIs described here are sampled regularly at the (instrument-dependent) detector sampling frequency and span the full science mission. They thus consist of a large amount of data. This section is concerned with the signal TOIs and their associated pointing TOIs. For selected detectors the DPCs provide a single timeline of cleaned and calibrated signal and three timelines of coordinates, corresponding to the two angular coordinates and one orientation angle.
Obviously these vectors are very long (about 1.38×1010 samples for HFI, from 2.5×106 to 5.5×106 for LFI) and thus need to be split into multiple files for export. Here the data are split by operational day (OD) as follows:
- HFI has 12 files per OD, one for pointing and one for signal for each channel, covering ODs 91 to 974, all sampled at Fsamp = 180.3737 Hz;
- LFI has six files per OD, three each for signal and pointing, since the sampling frequency is channel dependent, and covering ODs 91 to 1543, sampled at Fsamp = 32.5079, Fsamp = 46.5455, and Fsamp = 78.7692 Hz.
The signal timelines are encoded as single-precision real values, but the pointing vectors had to be encoded as double-precision reals to maintain the required accuracy. The result is that the total volume of the full dataset is approximately 30 TB. All files also contain the OBT.
The signal timelines have been cleaned of all known instrumental systematic effects, they have been calibrated in flux, and the solar and Earth motion dipole signals have been removed. But they have not been cleaned of the low-frequency, "1/f" noise that needs to be removed via a destriping tool. The methods consists of removing offsets or "baselines" determined by minimizing the differences in the signals at points on the sky where they intersect. For HFI an offset per ring is determined; for LFI the the baseline is computed every 0.246, 0.988, and 1.000 seconds for the 30, 44, and 70 GHz channels, respectively and maintain the same structure of the signal timelines.
In the case of HFI these offsets are determined using the full mission and all the valid detectors per channel, and they are then used for all the maps produced, i.e., for those maps using any fraction of the mission (year, survey) or any subset of the detectors (single detector, detector set), and it has been shown that using offsets determined from a limited part of the mission and/or a subset of the detectors yields maps that are less consistent than otherwise.
In the case of LFI these offsets are determined using the full mission and all the valid detectors per channel; those values have been used for the production of the full mission period maps. Note that baseline used for shorter period maps are determined on those data periods to avoid noise cross-correlation effects and those are not delivered.
The offsets are delivered separately, as described below.
Furthermore, for HFI and LFI, there are three sets of offsets produced: the primary set using the full rings; and the secondary ones using the first and second half rings, only. The differences between the primary and secondary sets are fairly minor, but they are necessary to include to reconstruct the maps as they were built by the HFI-DPC and LFI-DPC. In particular the offsets are useful for users wishing to build a map of a small area of the sky.
The HFI delivers its offsets in a ROI, or "ring-ordered Information" file. That file contains a table of Nrings = 26766 rows by Nbolometers in which each cell contains a 3-element vector containing the full-ring, first half-ring, and second half-ring offsets. The indices of the signal TOI delimiting the rings are given in a second ROI file with the same Nrings rows containing global parameters.
The LFI delivers its offsets in a TOI format, the structure is exactly the same as that used by the science timelines. The user can simply subtract one-by-one the offset timelines from the science timelines and then generate a map using the result. In the case of the half-ring baselines, a vector has been added in the OBT extension; this vector contains "1" or "2," depending which half ring it should be applied to.
HFI indexing[edit]
Each FITS file contains keywords giving the first and last index of the section of the full TOI contained in the file. These indices are those of the internal DPC representation; they begin at 0, corresponding to the switch-on of the instrument, and run to some very large number (about 25 billion for HFI) representing the full mission. Only the science part of the mission is exported, which is about 60% of the total for HFI. Left out are the early mission phase (cool-down and validation phases) and the warm phase of HFI, during which LFI continued to collect data. The HFI keywords also indicate which rings are included in each OD, but note that normally an OD does not begin at the same time (with the same index) as a ring.
Note that for historical reasons, the OD definitions of the two DPCs differ; for LFI they occur at the transition between pointing periods, whereas for HFI they do not. However, this has no significance for the user, since this splitting is somewhat arbitrary anyway, and what counts is the full vector once it is rebuilt in the user's own work space.
HFI timelines at the DPC are indexed from 0 to 25×109, which correspond to instrument switch-on and switch-off, respectively. Of these the indices 1.4×109 to 151.5×109 correspond to the science mission and are exported and delivered. Each file contains a keyword giving the first and last index of the data in that file, and EndIndex(OD)+1 = BeginIndex (OD+1). The ROI "Global" file gives the Begin/End Indices of each Ring, or Pointing Period, and can be used to destripe the signal TOIs with the offsets provided in the "Destriping-Offsets" ROI file.
Production process[edit]
HFI processing[edit]
Calibrated timelines[edit]
The input TOIs are in ADUs, representing the voltage signal at the end of the electronics (see Detection chain for details). The processing applied to remove instrumental effects and to calibrate them is described in detail in the HFI TOI processing section, and in Planck-2015-A07[3] we give a very brief summary here for convenience. This pipeline performs the following operations.
- ADC correction
- This corrects for the uneven size of the ADC bins (see ADC correction).
- Demodulation
- This is performed around a variable level, which is determined from the valid input data (using a validity flag from a previous version of the processing), and the data are converted to engineering units (V) using known conversion coefficients.
- Despiking
- The redundancy within a ring is used to determine where glitches occur. Once identified, the glitches are fitted with templates. A glitch flag is produced that identifies the strongest part of the glitches, and a timeline of glitch tails. produced from the template fits, is subtracted from the demodulated timeline. Finally, the flagged regions are replaced with data from an average over the pointing period in order to not leave holes in the timeline that would perturb the usage of Fourier methods on them
- Dark template removal
- The two dark bolometers are demodulated and despiked as above; the resulting timelines are then smoothed and used as an indicator of the overall temperature variations of the bolometer plate. Where the variations are consistent with each other, they are combined and removed from the bolometer signal timelines using appropriate coupling coefficients. The few percent of the data where they are not consistent are flagged as not valid on the timelines.
- Conversion to absorbed power
- The timeline is converted to watts of absorbed power using the bolometer function. This includes a nonlinearity correction and removal of the 4-K cooler lines (i.e., the electromagnetic interference of the 4-K cooler with the bolometer readout wires, which induces some sharp lines in the signal power spectra at frequencies of the 4-K cooler's fundamental and its multiples, folded by the signal modulations). Fourier coefficients of the relevant lines are determined on a per-ring basis, and then removed from the data. The quality of the removal depends on the bolometer.
- Deconvolution by the time transfer function
- This is done to correct for the non-instantaneous time response of the bolometers. The function itself is modeled using up to eight time constants, which are adjusted primarily on planet data and also from comparisons of the northward and southward scans of the Galactic Plane. It is then removed using Fourier techniques.
- Jump correction
- This corrects some rare jumps in the signal baseline (there are on average around 0.3 jumps per bolometer per pointing period). The jumps are detected and characterized on smoothed TOIs, and corrected by adding a constant to part of the signal timeline. The origin of the jumps is not known.
The results of this set of processing steps are a timeline of signal (in absorbed watts) and a "valid data" flag timeline for each of the 50 valid bolometers that are processed, out of the 52 HFI bolometers; these timelines contain the full-sky signal, i.e., including the solar and orbital dipoles, the Zodiacal light, and contributions from the far sidelobes. The dipoles are necessary for the flux calibration and are removed at the mapmaking stage. The remaining two bolometers (143-8 and 545-3) show semi-random jumps in the signal level, typically jumping over 2 to 5 different "pseudo-baseline" levels, a behaviour known as "random telegraphic signal" (RTS), so that these are commonly called the RTS bolometers. Finally, ring-level statistics of different types (mean, median, rms, kurtosis, etc.) are determined on a per-ring basis for all timelines, and a selection based on these statistics is used to discard anomalous rings, which are recorded in a ring-level flag for each valid bolometer timeline (see the Discarded rings section).
Throughout this processing, some planet signals are masked in the timeline in order to avoid ringing effects in the processing. This concerns only Jupiter at 100 and 143 GHz, Jupiter and Saturn at 217 GHz, and Jupiter, Saturn, and Mars at 353, 545, and 857 GHz (thanks to various improvements in the despiking and in the transfer functions it is no longer necessary to flag these three planets in the CMB channels, as was done in the 2013 release). Since they move on the sky, the portion of the sky masked during one survey is observed during another one, and no hole is left in the final map. In parallel, the planet data are processed in a similar way, but with different parameters for the despiking step. These results are processed separately to determine the beam shapes and the focal plane geometry.
Next, the TOIs are calibrated in astrophysical units using the results of the calibration pipeline (see the Map-making and calibration section) and the best estimates of the zero-point offsets (a constant level for each bolometer). These values are given in the RIMO. Also, the solar and Earth-motion dipole signals are computed and removed.
These TOIs are accompanied by several flags that are described below. The most important one is the "Total" flag, which identifies all the samples that were discarded by the DPC mapmaking, as described in HFI TOI processing. This flag includes the portion beyond 72 min for the rings that are longer; this is not used for science analysis because of the slight drift of the satellite pointing direction (spin axis) during these long acquisition periods.
At this point the TOIs still contain the low frequency (1/f) noise, which should be removed before projection onto a map. That cleaning step is called "destriping" or "baseline removal". The HFI-DPC does its destriping at ring-level, meaning that a constant is added to the signal of each ring in order to minimize the difference at the ring crossings, where the signal should be the same for all detectors (with the necessary precautions as described in Map-making and calibration section); should the user want to use the DPC's offsets, they are provided separately (see ROI files below). Maps produced from these TOIs, and after subtraction of the DPC's destriping offsets, are not identical to the maps that are delivered. This is discussed in Section A.2 of Planck-2015-A08[5].
As indicated above, the brightest planets are masked in the TOI in order to avoid ringing problems. For users specifically wanting to study these planets, we provide separate timelines covering just the planet transits. These timelines also include transits of Uranus and Neptune, which are not masked in the regular TOIs. They are produced with less agressive deglitching options in order to work on the rapidly changing baselines. These are the data used the reconstruction of the focal plane geometry and also for the determination of the scanning beams.
The table below lists the 16 transits, giving for each the begin/end ring number and the begin/end OD in which that ring is found. Note that transit 15 is split into two parts because some tests were done during the transit. The last row is a region processed in the same manner but without a planet transit, which is included for comparison.
Number | Beg Ring | End Ring | Beg OD | End OD | Comment |
---|---|---|---|---|---|
1 | 2700 | 3211 | 160 | 176 | Jupiter, Mars, Neptune |
2 | 3892 | 4099 | 203 | 212 | Uranus |
3 | 4575 | 4775 | 233 | 240 | Saturn |
4 | 7900 | 8150 | 330 | 337 | Mars |
5 | 8979 | 9188 | 367 | 376 | Neptune |
6 | 9550 | 9750 | 392 | 402 | Saturn |
7 | 9940 | 10300 | 411 | 425 | Jupiter, Uranus |
8 | 14018 | 14227 | 537 | 544 | Neptune |
9 | 14820 | 15190 | 567 | 583 | Jupiter, Uranus |
10 | 15925 | 16275 | 612 | 624 | Saturn |
11 | 20016 | 20225 | 734 | 743 | Neptune |
12 | 20900 | 21650 | 775 | 804 | Saturn, Uranus |
13 | 21780 | 22150 | 808 | 818 | Jupiter |
14 | 24992 | 25202 | 916 | 921 | Neptune |
15a | 25830 | 25849 | 937 | 938 | Mars, Uranus |
15b | 25864 | 26225 | 947 | 957 | Mars, Uranus |
16 | 26650 | 27005 | 968 | 974 | Jupiter |
17 | 12000 | 12150 | 479 | 483 | background |
The pointing (see common Detector Pointing section for details) is determined from the AHF produced by the MOC, which gives the direction and orientation of the LOS of a fiducial position in the focal plane at frequencies of 8Hz during stable pointing and 4Hz during manoeuvres. This is interpolated to the timestamps of data samples, corrected for the wobble and other time-dependent offsets determined from the observed positions of a large number of sources around the sky, and finally converted to the LOS of each detector using the quaternions in the IMO. These pointing data are valid for both the regular and the planet TOIs.
Semi-raw timelines[edit]
The only step which has been applied to the HFI semi-raw TOIs is the ADC non-linearity correction (to first order), which includes some level of 4k-line removal. The semi-raw timelines are in units of Volts and have not been de-modulated.
LFI processing[edit]
Calibrated timelines[edit]
The input TOIs are in ADUs, representing the voltage signal at the output of the electronics (see Radiometer Chain Assembly (RCA) for details). The processing applied to remove instrumental effects and to calibrate them is described in detail in the LFI TOI processing section, in Planck-2015-A03[1] and in Planck-2020-A2[2]; here we give a very brief summary for convenience. That pipeline performs the following operations.
- ADC correction
- Due to ADC nonlinearity under certain condition, this instrumental effect is removed by applying well know templates directly to the diode signal.
- Electronics spikes
- This is caused by the interaction between the electronics clock and the scientific data lines. The signal is detected in all the LFI radiometer time-domain outputs as a 1-s square wave with a rising edge near 0.5s and a falling edge near 0.75s, synchronous with the onboard time signal. In the frequency domain it appears as a spike signal at multiples of 1Hz. The 44-GHz channels are the only LFI outputs significantly affected by this effect. Consequently the spike signal is removed from the data only in these channels.
- Demodulation
- Each LFI diode switches at 4096Hz between the sky and the 4-K reference load. The data acquired in this way are dominated by 1/f noise that is highly correlated between the two streams; differencing those streams results in a strong reduction of 1/f noise. The procedure applied is described in LFI TOI processing, taking into account that the gain modulation factor R was computed on timestreams with the Galaxy and point sources masked to avoid strong sky signals.
- Diode combination
- Two detector diodes provide the output for each LFI receiver channel. To minimize the impact of imperfect isolation of the data stream of each diode, we perform a weighted average of the time-ordered data from the two diodes of each receiver. The procedure applied is detailed in LFI TOI processing; and the weights used are kept fixed for the entire mission.
- Scientific calibration
- This step calibrates the timelines to physical units KCMB, fitting the total CMB dipole convolved with the 4π beam representation, without taking into account the signature due to Galactic stray light.
- Gain regularization
- The calibration constants computed using the model of the dipole signal suffer from large uncertainties when the Planck spacecraft is badly aligned with the dipole axis. To reduce the noise, we apply an adaptive smoothing algorithm, which is also designed to preserve the discontinuities caused by abrupt changes in the working configuration of the radiometers (e.g., sudden temperature changes in the focal plane).
- Removal of solar and orbital dipole signal
- The combined solar and orbital dipole is convolved with the 4π beam representation of each radiometer and then removed from its timeline.
- Removal of Galactic stray light
- The light incident on the focal plane without reflecting on the primary mirror (stray light) is a major source of systematic effects, especially when the Galactic plane intersects the direction of the main spillover. This effect is corrected by removing the estimated straylight signal from the timelines. This signal is computed as the convolution of Galaxy simulation with the beam sidelobes (see details in LFI TOI processing).
After these processing steps the timelines are used for the production of the maps.
The pointing (see common Detector Pointing section for details) is determined from the AHF produced by the MOC, which gives the direction and orientation of the LOS of a fiducial position in the focal plane at frequencies of 8Hz during stable pointing and 4Hz during manoeuvres. This is interpolated to the timestamps of data samples, corrected for the wobble and other time-dependent offsets determined from the observed positions of a large number of sources around the sky, as well as temperature sensors, and finally converted to the LOS of each detector.
Semi-raw timelines[edit]
The steps which have been applied to the LFI semi-raw TOIs are: ADC correction, spike removal, and demodulation with R-factor application. The semi-raw timelines are in units of ADUs.
File Names[edit]
The file names are of the form
{H,L}FI_TOI_{fff}-{SCI,RAW,PTG,TAI,OFF}_Rn.nn_ODxxxx.fits,
where
- fff denotes the frequency,
- SCI denotes calibrated signal TOIs,
- RAW denotes semi-raw signal TOIs,
- PTG denotes pointing TOIs,
- TAI denotes the OBT-MJD correlation TOI,
- OFF denotes the baseline TOI,
- Rn.nn is the version, and
- ODxxxx indicates the OD.
Regarding the HFI TOIs for the 100 to 217GHz channels, the R2.00, made public in Jan 2015, contained only the unpolarized bolometer timelines, while R2.02, made public in July 2015, contains all bolometers.
The HFI planet timelines are named
{H,L}FI_TOI_{fff}-SCI-planets_R2.nn_ODxxxx.fits
and are provided for all 50 valid bolometers, i.e., SWBs and PSBs.
FITS file structure[edit]
All FITS files begin with a primary extension containing no data and a minimal header, which is followed by one or more BINTABLE extensions containing data, and with a description of the data in the header keywords. In what follows we will usually ignore the primary extension, and count only the extensions containing data.
TOI files[edit]
The signal FITS files contain N+1 , "BINTABLE", data extensions, where N is the number of detectors in that frequency channel. The first extension contains the OBT and its flags, called the "global" flags since they apply to all detectors, which is followed by one extension for each detector, containing the signal and its "local" flag. The flags columns are written as "byte" in which each of the 8 bits (maximum) encodes one flag timeline. The meaning of each bit is given in the header comments and is subject to change. For the global flag they include the following list.
- For HFI:
- unstable pointing, where 1 = pointing is not stable (e.g., during repointing manoeuvres);
- dark correlation, where 1 = darks are uncorrelated and data are flagged;
- first/Second half ring, which samples are in which half (only covers the stable pointing part of the ring);
- HCM, whether in HCM mode (unstable pointing);
- and more (see extension header for details).
- For LFI:
- bit 0, unstable pointing, where 1 = pointing is not stable;
- bit 1, time correlation quality, where 1 = outside specification;
- bit 2, special observation, where 1 = special observation, such as a deep scan.
For the local flag they include the following list.
- For HFI:
- total flag, being a combination of the various flags that is the one finally used in the mapmaking (all samples with "total flag" different from zero should not be used);
- data not valid, e.g., glitched samples;
- despike common (for PSBs only), corresponding to a glitch on the current or the other of a PSB pair;
- strong signal, i.e., on the Galactic plane;
- strong source, i.e., on a point source;
- and more (see extension header for details).
- For LFI:
- bit 0, data not valid, where 1 = science sample not valid;
- bit 2: planet crossing, where 1 = science sample containing planet;
- bit 3: moving objects, where 1 = minor Solar System object (not yet used);
- bit 4: gap, where 1 = this sample was artificially included due to a gap in the data.
For HFI the header extension gives more details on the flags and the meaning of "1" and "0".
1. EXTNAME = "OBT" : Data columns | ||||
---|---|---|---|---|
Column name | Data type | Units | Description | Comment |
OBT | Double | 2-16 sec | Onboard time | |
FLAG | Byte | None | Various bit-level flags | |
Keyword | Data Type | Value | Description | Comment |
OD | Int | OD covered (as in filename) | ||
BEGIDX | Int | First sample index of given OD | Only HFI | |
ENDIDX | Int | Last sample index of given OD | Only HFI | |
BEGRING | Int | First ring in given OD | Only HFI | |
ENDRING | Int | Last ring in given OD | Only HFI | |
TIMEZERO | String | 1958-01-01z00:00 | Origin of OBT | Only HFI |
TIMEZERO | Float | 106744000000000. | Origin of OBT | Only LFI |
n. EXTNAME = DETNAME : Data columns | ||||
Column name | Data type | Units | Description | Comment |
SIGNAL | Real*4 | Kcmb or MJy.sr-1 | Value of signal | Comment |
FLAG | Byte | None | Various bit-level flags | |
Keyword | Data type | Value | Description | Comment |
UNIT | String | Units of signal | Only HFI, LFI always Kcmb | |
DESTRIPE | 1/0 | Whether timeline is destriped | Only HFI | |
OD | Int | OD covered (as in filename) | ||
BEGIDX | Int | First sample index of given OD | Only HFI | |
ENDIDX | Int | Last sample index of given OD | Only HFI | |
BEGRING | Int | First ring in given OD | Only HFI | |
ENDRING | Int | Last ring in given OD | Only HFI |
The HFI Planet files have the same structure, but the local flags contain a single "Data not valid" flag. Also, the first and last ODs of each transit are usually not complete, since they contain only the rings that are included in the transit observations.
The pointing files have a similar structure, except that DETNAME extensions contain three columns of Real*8 variables with the φ, θ, and ψ Galactic spherical coordinates of each sample in radians, as shown in the table below. There is no local flag for the coordinates.
n. EXTNAME = DETNAME : Data columns | |||
---|---|---|---|
Column name | Data type | Units | Description |
PHI | Real*8 | Radian | Longitude |
THETA | Real*8 | Radian | Colatitude |
PSI | Real*8 | Radian | Roll angle |
HFI ROI files[edit]
The files provided by HFI are
- HFI_ROI_GlobalParams_RelNum_full.fits,
- HFI_ROI_DestripeOffsets_RelNum_full.fits,
which are described below.
- Global parameters
- This ROI file contains a single "BINTABLE" extension with three columns that give, for each ring, the index number of the first sample, the ESA pointing ID of the ring, and the ring start time. All three are encoded as Int*8, and the time is given in units of 2-16 sec from "TIMEZERO", 1958-01-01z00:00, as in the TOI files. Rings are numbered from 240 to 27005, as given in the "BEGRING" and "ENDRING" keywords.
- Destriping offsets
- This ROI file contains a single "BINTABLE" extension with 3 columns that give, for each ring, the offsets to subtract from each bolometer signal timeline or vector. There are three offsets available for each bolometer (full-ring, first half-ring, and second half-ring), for building the full-ring or the two half-ring maps, respectively. The offsets must be subtracted from Index(N) to Index(N+1)-1 of the corresponding signal timeline, where N is the ring number, and the indices are given in the "global" parameters file. The offsets are given in the same units as the signal vectors, that is Kcmb for the 100- to 353-GHz channels, and MJy.sr-1 for the 545- and 857-GHz channels. All values are encoded as Real*4. The rings for which the offsets are set to 0.000 are "badrings" which are rejected in the mapmaking process. These rings are also flagged by the "TotalFlag", as are the portions of rings longer than about 72.5 min, where the drift in the satellite's spin axis becomes important.
TAI TOI files[edit]
The TAI TOI files contain one extension with two columns, the first is the OBT value (exactly the same as reported in the SCI TOI), the second is the corresponding modified Julian day. Note that leap seconds were not added.
1. EXTNAME = "OBT-MJD" : Data columns | ||||
---|---|---|---|---|
Column name | Data type | Units | Description | Comment |
OBT | Int*8 | 2-16 sec | Onboard time | |
MJD | Real*8 | day | Modified Julian day |
LFI OFF TOI files[edit]
The OFF (baseline) TOI files adopt the same file structure as the Science TOI files. Note that in case of OFF timelines related to the half-ring, an additional column is included in the OBT extension to define for each sample if it belongs to half-ring 1 or half-ring 2.
LFI HouseKeeping files[edit]
House keeping timelines are:
- LFI_TOI_DAE-FAST-HK_R2.nn_ODxxxx.fits;
- LFI_TOI_DAE-SlowConfiguration-HK_R2.nn_ODxxxx.fits;
- LFI_TOI_DAE-SlowCurrent-HK_R2.nn_ODxxxx.fits;
- LFI_TOI_DAE-SlowPhaseSwitch-HK_R2.nn_ODxxxx.fits;
- LFI_TOI_DAE-SlowVoltage-HK_R2.nn_ODxxxx.fits;
- LFI_TOI_REBA-HK_R2.nn_ODxxxx.fits;
- SCS_TOI_EssentialASW-HK_R2.nn_ODxxxx.fits;
- SCS_TOI_NonEssentialASW-HK_R2.nn_ODxxxx.fits.
Each file contains two extensions, the first is the OBT (values are sampled every 1 or 10 seconds), while the second contains a variable number of columns equivalent to twice the number of Housekeeping timelines stored. Each Housekeeping datum is accompanied by its flag (normally "0" means that the value was considered invalid or out of limits). The Housekeeping names are the ones defined in the LFI Instrument Operation Manual.
Other Releases: 2020 NPIPE[edit]
2020 Release NPIPE timelines
The NPIPE release includes pointing information, as well as three flavours of timelines for both Planck instruments.
Pointing[edit]
NPIPE satellite attitude files are interpolated to detector time-stamps. Since all HFI detectors are sampled at the same rate, four versions cover all Planck sampling rates: 30GHz; 44GHz; 70GHz; and HFI frequencies. Unlike earlier releases, NPIPE pointing files include the low-frequency pointing corrections (PTCOR), so only aberration corrections are needed to reconstruct detector pointing.
Individual detector pointing is also supplied to support PLA facilities that require it. Given the large volume of the individual detector pointing files, most users are better served by the satellite pointing files.
Raw timelines[edit]
The raw timelines contain the detector signal with minimal processing. For LFI this means four separate timelines for each radiometer assembly, consisting of a sky and a load stream for each of the two diodes. LFI preprocessing collapses them into a single timeline for each radiometer.
Raw HFI timelines were corrected with the initial ADCNL profiles solved from warm HFI data acquired during the LFI extended mission. They include the HFI dark bolometer timelines that serve a similar function as the LFI load signal.
Preprocessed timelines[edit]
Preprocessed timelines are approximately calibrated to CMB kelvins. They are jump-corrected and anomalous pointing periods are flagged. LFI timelines are collapsed to a single timeline per radiometer and ADCNL-corrected using DPC-provided ADCNL profiles.
HFI timelines are decorrelated using dark bolometer data. Cosmic-ray glitches are located, subtracted and flagged. Bolometric nonlinearity is corrected for using ground-measured nonlinearity models.
Reprocessed timelines[edit]
Reprocessed timelines have been bandpass-mismatch and far sidelobe-corrected. Gain fluctuations have been corrected for by fitting a step-wise gain-fluctuation model. For HFI, the second-order ADCNL was measured and corrected for in the 100-217GHz channels. All these template corrections are solved on full-mission, full-frequency data. Residual errors are highly correlated, so it is not advised to split the reprocessed timelines to make, e.g., detector-set or half-mission maps.
The timelines were destriped with the Madam destriper code to suppress 1/f noise. For better flexibility, the 1/f noise was solved using only a single detector, depolarized signal. Small-scale, high-frequency noise in the timelines is approximately independent of other detectors, although secondary mechanisms (co-incident glitches or errors in the polarization template) introduce weak correlations.
File Names[edit]
Most file names are of the form
{H,L}FI_TOI_{fff}-{RAW,preprocessed,reprocessed,PTG}_Rn.nn_ODxxxx.fits,
where
- fff denotes the frequency,
- RAW denotes raw signal TOIs,
- preprocessed denotes preprocessed TOIs,
- reprocessed denotes reprocessed TOIs,
- PTG denotes detector pointing TOIs,
- Rn.nn is the version, and
- ODxxxx indicates the OD.
Satellite pointing files are of the form
satellite_pointing_{030,044,070,HFI}_Rnnn_ODxxxx.fits.
FITS file structure[edit]
All FITS files begin with a primary extension containing no data and a minimal header, which is followed by one or more BINTABLE extensions containing data, and with a description of the data in the header keywords. In what follows we will usually ignore the primary extension, and count only the extensions containing data.
TOI files[edit]
The signal FITS files contain N+1 , "BINTABLE", data extensions, where N is the number of detectors in that frequency channel. The first extension contains the OBT and its flags, called the "global" flags since they apply to all detectors, which is followed by one extension for each detector, containing the signal and its "local" flag. The flags columns are written as "byte" in which each of the 8 bits (maximum) encodes one flag timeline. The meaning of each bit is given in the header comments and is subject to change. For the global flag they are as follows.
- For HFI RAW files:
- unwritten, 1 = timestamps NOT written;
- unstable pointing, where 1 = pointing is not stable (e.g., during repointing manoeuvres).
- For LFI RAW files:
- bit 0, unstable pointing, where 1 = pointing is not stable.
For the local flag they include the following list.
- For LFI and HFI RAW data:
- bit 0, data not valid, where 1 = science sample not valid.
Other flag bits are defined and explained in the file header but not used in NPIPE processing.
1. EXTNAME = "OBT" : Data columns | ||||
---|---|---|---|---|
Column name | Data type | Units | Description | Comment |
OBT | 64-bit integer | 2-16 sec | Onboard time | LFI |
OBT | Double | TAI nanosec | Onboard time | HFI |
FLAG | Byte | None | Various bit-level flags | |
Keyword | Data type | Value | Description | Comment |
BEGIDX | Int | First sample index in file | Only HFI | |
ENDIDX | Int | Last sample index in file | Only HFI | |
n. EXTNAME = DETNAME : Data columns | ||||
Column name | Data type | Units | Description | Comment |
SKY | 64-bit float | ADU | Value of decompressed sky signal | LFI |
REF | 64-bit float | ADU | Value of decompressed reference load signal | LFI |
SIGNAL | 32-bit integer | ADU | Value of ADCNL-corrected signal | HFI |
FLAG | Byte | None | Various bit-level flags |
1. EXTNAME = "OBT" : Data columns | ||||
---|---|---|---|---|
Column name | Data type | Units | Description | Comment |
OBT | 64-bit integer | 2-16 sec | Onboard time | LFI |
OBT | Double | TAI nanosec | Onboard time | HFI |
FLAG | Byte | None | Various bit-level flags | |
Keyword | Data Type | Value | Description | Comment |
BEGIDX | Int | First sample index in file | Only HFI | |
ENDIDX | Int | Last sample index in file | Only HFI | |
n. EXTNAME = DETNAME : Data columns | ||||
Column name | Data type | Units | Description | Comment |
SIGNAL | Double | KCMB | Value of (p)reprocessed signal | |
FLAG | Byte | None | Various bit-level flags |
1. EXTNAME = "OBT" : Data columns | ||||
---|---|---|---|---|
Column name | Data type | Units | Description | Comment |
OBT | 64-bit integer | 2-16 sec | Onboard time | LFI |
OBT | Double | TAI nanosec | Onboard time | HFI |
FLAG | Byte | None | Unvalid pointing flag |
|
2. EXTNAME = "spin_phase" : Data columns | ||||
Column name | Data type | Units | Description | Comment |
PHASE | Double | Degrees | Spacecraft spin phase | |
FLAG | Byte | None | Bad data flag |
|
3. EXTNAME = "attitude" : Data columns | ||||
Column name | Data type | Units | Description | Comment |
QUATERNION_X | Double | None | Spacecraft attitude quaternion X-component | |
QUATERNION_Y | Double | None | Spacecraft attitude quaternion Y-component | |
QUATERNION_Z | Double | None | Spacecraft attitude quaternion Z-component | |
QUATERNION_S | Double | None | Spacecraft attitude quaternion S-component | |
FLAG | Byte | None | Bad data flag |
|
4. EXTNAME = "velocity" : Data columns | ||||
Column name | Data type | Units | Description | Comment |
X_VEL | Double | km s-1 | Spacecraft barycentric velocity X-component | |
Y_VEL | Double | km s-1 | Spacecraft barycentric velocity Y-component | |
Z_VEL | Double | km s-1 | Spacecraft barycentric velocity Z-component | |
FLAG | Byte | None | Bad data flag |
|
5. EXTNAME = "position" : Data columns | ||||
Column name | Data type | Units | Description | Comment |
X_POS | Double | AU | Spacecraft barycentric position X-component | HFI only |
Y_POS | Double | AU | Spacecraft barycentric position Y-component | HFI only |
Z_POS | Double | AU | Spacecraft barycentric position Z-component | HFI only |
FLAG | Byte | None | Bad data flag |
|
References[edit]
- ↑ 1.01.1 Planck 2015 results. II. LFI processing, Planck Collaboration, 2016, A&A, 594, A2.
- ↑ 2.02.1 Planck 2018 results. II. Low Frequency Instrument data processing, Planck Collaboration, 2020, A&A, 641, A2.
- ↑ 3.03.1 Planck 2015 results. VII. High Frequency Instrument data processing: Time-ordered information and beam processing, Planck Collaboration, 2016, A&A, 594, A7.
- ↑ Planck 2018 results. III. High Frequency Instrument data processing and frequency maps, Planck Collaboration, 2020, A&A, 641, A3.
- ↑ Planck 2015 results. VIII. High Frequency Instrument data processing: Calibration and maps, Planck Collaboration, 2016, A&A, 594, A8.
(Planck) Low Frequency Instrument
(Planck) High Frequency Instrument
Planck Legacy Archive
Operation Day definition is geometric visibility driven as it runs from the start of a DTCP (satellite Acquisition Of Signal) to the start of the next DTCP. Given the different ground stations and spacecraft will takes which station for how long, the OD duration varies but it is basically once a day.
On-Board Time
Data Processing Center
Ring-Ordered Information (DMC group/object)
Flexible Image Transfer Specification
analog to digital converter
random telegraphic signal
Cosmic Microwave background
reduced IMO
Attitude History File
[ESA's] Mission Operation Center [Darmstadt, Germany]
Line Of Sight
Angular momentum Control Mode
European Space Agency
LFI Data Acquisition Electronics
House Keeping
LFI Radiometer Electronics Box Assembly
Sorption Cooler Subsystem (Planck)
sudden change of the baseline level inside a ring