Difference between revisions of "Pre-processing"

From Planck Legacy Archive Wiki
Jump to: navigation, search
 
(24 intermediate revisions by 4 users not shown)
Line 2: Line 2:
  
 
In terms of data processing, the HFI ground segment handles two types of data, both made available via the MOC:
 
In terms of data processing, the HFI ground segment handles two types of data, both made available via the MOC:
* telemetry data transmitted from the satellite. These come from the different subsystems of the satellite service module, from the sorption cooler and from the two instruments.
+
* telemetry data transmitted from the satellite, coming from the different subsystems of the satellite service module, from the sorption cooler, and from the two instruments;
* auxiliary data. These are data produced by MOC. The only 3 products used by the HFI DPC are the pointing data, the orbit data, and the time correlation data.
+
* auxiliary data, produced by the MOC, with the only three products used by the HFI DPC being the pointing data, the orbit data, and the time correlation data.
All data are retrieved by the HFI level 1 software and stored in the HFI database.
+
All data are retrieved by the HFI Level 1 software and stored in the HFI database.
  
 
==Telemetry data==
 
==Telemetry data==
  
The digitized data from the satellite are assembled on board in packets according to the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations and the ESA Packet Utilization Standard. The packets are dumped to the ground during the Daily Tranmission Control Period, consolidated by and stored at MOC. Telemetry data contain the housekeeping data and the bolometer (ie science) data.
+
The digitized data from the satellite are assembled on board in packets, according to the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations, and the ESA Packet Utilization Standard. The packets are dumped to the ground during the Daily Tranmission Control Period, consolidated by and stored at the MOC. Telemetry data contain the housekeeping data and the bolometer (i.e., science) data.
  
 
===Housekeeping data===
 
===Housekeeping data===
  
For several reasons (systems monitoring, potential impact of the environment, understanding of the bolometer data), the HFI level 1 software gathers and stores in its database the satellite subsystems housekeeping parameters:
+
For several reasons (systems monitoring, potential impact of the environment, understanding of the bolometer data), the HFI Level 1 software gathers and stores in its database the satellite subsystems housekeeping parameters:
* Command and Data Management System
+
* Command and Data Management System;
* Attitude Control & Measurement Subsystem
+
* Attitude Control & Measurement Sub-system;
* Thermal Control System
+
* Thermal Control System;
* Sorption Cooler System
+
* Sorption Cooler System;
* HFI housekeeping parameters
+
* HFI housekeeping parameters.
The structure and the frequency of the packets built by these subsystems and the format of the house keeping parameters are described in the Mission Information Bases. The HFI L1 software uses these MIBs to extract the house keeping parameters from the packets. Given the status of each subsystem, the parameters are gathered in the HFI database in groups. The HFI L1 software builds in each group a vector of time (usually named TIMESEC) and a single vector per housekeeping parameter.
+
The structure and the frequency of the packets built by these subsystems and the format of the housekeeping parameters are described in the Mission Information Bases (MIBs). The HFI L1 software uses these MIBs to extract the housekeeping parameters from the packets. Given the status of each subsystem, the parameters are gathered in the HFI database in groups. The HFI L1 software builds in each group a vector of time (usually named TIMESEC) and a single vector per housekeeping parameter.
  
 
===Bolometer data===
 
===Bolometer data===
The HFI science data is retrieved and reconstructed as described in section 3.1 of <cite>#planck2011-1-7</cite> {{PEarly|6|page=5}}.
+
The HFI science data are retrieved and reconstructed as described in section 3.1 of {{PlanckPapers|planck2011-1-7}}.
  
The first stage of HFI data processing is performed on-board in order to generate the telemetry. This is described in the [[HFI_detection_chain#Data_compression]] section.
+
The first stage of HFI data processing is performed onboard in order to generate the telemetry. This is described in the [[HFI_detection_chain#Data_compression|data compression]] section.
  
On board, the signal from the 72 HFI channels is sampled at 180.4Hz by the Read-out Electronic Unit. 254 samples per channel are grouped into a compression slice. The Data Processing Unit then builds a set of several telemetry packets containing the compression slice data and adds to the first packet the start time of the compression slice. When receiving this set of telemetry packets, the L1 software extracts the <math>72 × 254</math> samples and computes the time of each sample based on the compression slice start time (digitized with <math>2^{-16} s = 15.26</math> μs quantization steps) and a mean sample time between samples. For the nominal instrument configuration, the sample integration time is measured to be Tsamp = 5.54404 ms
+
On board, the signal from the 72 HFI channels is sampled at 180.4Hz by the Read-out Electronic Unit. 254 samples per channel are grouped into a compression slice. The Data Processing Unit then builds a set of several telemetry packets containing the compression slice data and adds to the first packet the start time of the compression slice. When receiving this set of telemetry packets, the L1 software extracts the 72 &times; 254 samples and computes the time of each sample based on the compression slice start time (digitized with 2<sup>-16</sup> s = 15.26&mu;s quantization steps) and a mean sample time between samples. For the nominal instrument configuration, the sample integration time is measured to be <i>T</i><sub>samp</sub> = 5.54404ms.
  
The two plots show the histogram of time differences between two successive samples for the full mission. 6 sample intervals (> 6 ms) correspond to 3 occurrences of packets lost on board when the on board sofwtare of the Command and Data Management Unit has been patched (18, 19 and 20th of August 2009). The right plot is a blow up of the left one. It shows the distribution around the mean value of 5,54 ms and the 15 μs quantization step.
+
The two plots below show the histogram of time differences between two successive samples for the full mission. Six sample intervals (> 6 ms) correspond to three occurrences of packets lost on board when the on-board software of the Command and Data Management Unit was patched (18, 19, and 20 of August 2009). The right plot is a blow-up of the left one. It shows the distribution around the mean value of 5.54ms and the 15&mu;s quantization step.
  
 
{|table border=0 align=center cellpadding="3" cellspacing="0" style=text-align:center
 
{|table border=0 align=center cellpadding="3" cellspacing="0" style=text-align:center
Line 34: Line 34:
  
 
===Transfer functions===
 
===Transfer functions===
In order to ease the reading data in the HFI database, so called transfer functions are created. They allow software items to read data on which functions are applied on the fly.
+
In order to ease the reading of data in the HFI database, so called transfer functions are created. They allow software items to read data onto which functions are applied on the fly.
A simple example of a transfer function is the conversion of a thermometer data in Analogic to Digital Units (ADU) to Kelvins.
+
A simple example of a transfer function is the conversion of thermometer data in analogue-to-digital Units (ADUs) to kelvins.
Here we show the 2 transfer functions applied on the bolometer samples in the data processing pipelines:
+
Here we show the two transfer functions applied to the bolometer samples in the data processing pipelines.
  
  
'''From raw signal to non demodulated signal in Volts''' (transf1_nodemod)
+
'''From raw signal to non-demodulated signal in volts''' (transf1_nodemod):
  
<math>Signal\_{N\!D}\_V_{bc} = \dfrac{\dfrac{C_{bc}}{N_{sample} - Nblanck_{bc}}- offset_{bc}}{F1_{bc}} \times \dfrac{Gamp_{ETAL}}{Gamp_{bc}} </math>
+
<math>Signal\_{N\!D}\_V_{bc} = \frac{\frac{C_{bc}}{N_{\rm sample} - N_{\rm blanck,} bc}- O_{bc}}{F1_{bc}} \times \frac{G_{\rm amp, ETAL}}{G_{{\rm amp}, bc}} </math>.
  
where:
+
The different symbols are described below.
* <math>bc</math> refers to a bolometer usually labelled by its electronic belt and channel. Eg. bc=00 refers to the ''a'' part of the first 100 GHz Polarized Sentivite Bolometer.   
+
* <i>bc</i> refers to a bolometer usually labelled by its electronic belt and channel,  e.g., <i>bc=00</i> refers to the ''a'' part of the first 100GHz polarization sensitive bolometer.   
* <math>C_{bc}</math> is the bolometer sample in Analogic Digital Units (ADU)
+
* <i>C</i><sub>bc</sub> is the bolometer sample in ADU.
* <math>N_{sample}</math> is the number of samples in half a modulation period. This parameter is common to all bolometer channels and - although kept fixed during the whole mission - is read from the housekeeping parameter. <math>N_{sample}=40</math>
+
* <i>N</i><sub>sample</sub> is the number of samples in half a modulation period. This parameter is common to all bolometer channels and (although kept fixed during the whole mission) is read from the housekeeping parameter. <i>N</i><sub>sample</sub>=40.
* <math>Nblanck_{bc}</math> is the number of samples suppressed at the beginning oft each half modulation period. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For bc=00, <math>Nblanck_{00}=0</math>  
+
* <i>N</i><sub>blanck, bc</sub> is the number of samples suppressed at the beginning of each half-modulation period. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For <i>bc=00</i>, <i>N</i><sub>blanck, 00</sub>=0.
* <math>F1_{bc}</math> is a calibration factor. For bc=00, <math>F1_{00} \simeq 1.8\times10^7</math>.
+
* <i>F1</i><sub>bc</sub> is a calibration factor. For <i>bc=00</i>, <i>F1</i><sub>00</sub>&asymp;1.8&times;10<sup>7</sup>.
* <math>offset_{bc}</math> is close to 32768
+
* <i>O</i><sub>bc</sub> is close to 32768.
* <math>Gamp_{ETAL}</math> is the gain amplifier mesured during the calibration phase. For bc=00, <math>Gamp_{ETAL}=1</math>   
+
* <i>G</i><sub>amp, ETAL</sub> is the gain amplifier measured during the calibration phase. For <i>bc=00</i>, <i>G</i><sub>amp,ETAL</sub>=1.  
* <math>Gamp_{bc}</math> is the current gain amplifier. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For bc=00, <math>Gamp_{00}=1</math>
+
* <i>G</i><sub>amp, <i>bc</i></sub> is the current gain amplifier. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For <i>bc=00</i>, <i>G</i><sub>amp,00</sub>=1.
  
  
'''From raw signal to demodulated signal in Volts''' (transf1)
+
'''From raw signal to demodulated signal in Volts''' (transf1):
  
 +
the same formula as above is used, but with the demodulation and a 3-point filter computed as
  
The same formula as above is used but with the demodulation and a 3-point filter computed as:
+
<math>Signal\_V_{bc} [n] = - \frac{1}{4} (-1)^{{\rm parity}_{n-1}} Signal\_{N\!D}\_V_{bc} [n-1] + \frac{1}{2} (-1)^{{\rm parity}_{n}} Signal\_{N\!D}\_V_{bc} [n] - \frac{1}{4} (-1)^{{\rm parity}_{n+1}} Signal\_{N\!D}\_V_{bc} [n+1]</math>,
  
<math>Signal\_V_{bc} [n] = - \tfrac{1}{4} (-1)^{parity_{n-1}} Signal\_{N\!D}\_V_{bc} [n-1] + \tfrac{1}{2} (-1)^{parity_{n}} Signal\_{N\!D}\_V_{bc} [n] - \tfrac{1}{4} (-1)^{parity_{n+1}} Signal\_{N\!D}\_V_{bc} [n+1]</math>
+
where
 
+
* <i>n-1</i> and <i>n+1</i> refer to the samples before and after the given <i>n</i> sample to demodulate,
where:
+
* "parity" is computed by the HFI L1 software.
* <math>n-1</math>, and <math>n+1</math> refer to the samples before and after the given <math>n</math> sample to demodulate
 
* <math>parity</math> is computed by the HFI L1 software
 
  
 
===Statistics on the telemetry data===
 
===Statistics on the telemetry data===
  
The table gives some statistics about the data handled at the pre-processing level:
+
The following table gives some statistics about the data handled at the pre-processing level.
  
{|table border=1 align=center cellpadding="3" cellspacing="0" style=text-align:center
+
<center>
! - !! Nominal mission<sup>(1)</sup> !! Full mission<sup>(1)</sup> !! From launch to the end of full mission<sup>(1)</sup>
+
{| class="wikitable"  align="center" style="text-align:center" border="1" cellpadding="5" cellspacing="0"
 +
|- bgcolor="ffdead"
 +
! || Nominal mission<sup>(1)</sup> || Full mission<sup>(1)</sup> || From launch to the end of full mission<sup>(1)</sup>
 
|-
 
|-
!align="left"|Duration
+
|align="left"|Duration|| 473 days ||884 days || 974 days  
|align="center"|<math>473</math> days ||align="center"|<math>884</math> days || align="center"|<math>974</math> days  
 
 
|-
 
|-
!align="left"|Number of HFI packets generated onboard (HSK/science)<sup>(2)</sup>
+
|align="left"|Number of HFI packets generated onboard (HSK/science)<sup>(2)</sup>|| 19 668 436 / 376 294 615  ||  37 762 493 / 704 852 262 || 41 689 4090 / 765 043 713
|   <math>19~668~436</math> / <math>376~294~615</math> ||  <math>37~762~493</math> / <math>704~852~262</math> || <math>41~689~4090</math> / <math>765~043~713</math>
 
 
|-
 
|-
!align="left"|Number of HFI packets lost <sup>(3)</sup> (HSK/science)
+
|align="left"|Number of HFI packets lost <sup>(3)</sup> (HSK/science) || 2 / 20 || 2 / 20 || 2 / 20
| align="center"|<math>2</math> / <math>20</math> || align="center"|<math>2</math> / <math>20</math> || align="center"|<math>2</math> / <math>20</math>
 
 
|-
 
|-
!align="left"|ratio of HFI lost packets vs generated on board (HSK/science)
+
|align="left"|Ratio of HFI lost packets to those generated on board (HSK/science) || 1&times;10<sup>-7</sup> / 5&times;10<sup>-8</sup> || 5&times;10<sup>-8</sup> / 3&times;10<sup>-8</sup>|| 5&times;10<sup>-8</sup> / 3&times;10<sup>-8</sup>  
| align="center"|<math>1\times10^{-7}</math> / <math>5\times10^{-8}</math> || align="center"|<math>5\times10^{-8}</math> / <math>3\times10^{-8}</math>||align="center"|<math>5\times10^{-8}</math> / <math>3\times10^{-8}</math>  
 
 
|-
 
|-
!align="left"|Number of different housekeeping parameters stored in the database (HFI/SCS/sat)
+
|align="left"|Number of different housekeeping parameters stored in the database (HFI/SCS/sat)||colspan="3"  align="center"| 3174 / 708 / 12390
|colspan="3"  align="center"|<math>3174</math> / <math>708</math> / <math>12390</math>
 
 
|-
 
|-
!align="left"|Number of science samples stored in the database
+
|align="left"|Number of science samples stored in the database ||530 632 594 653|| 991 929 524 565 || 1 090 125 748 960
| align="center"|<math>530~632~594~653</math>|| align="center"|<math>991~929~524~565</math> || align="center"|<math>1~090~125~748~960</math>
 
 
|-
 
|-
!align="left"|Number of missing science samples
+
|align="left"|Number of missing science samples || 2 537 499 || 6 634 491<sup>4</sup> || 7 521 758  
| align="center"|<math>2~537~499</math> || align="center"|<math>6~634~491</math><sup>(4)</sup> || align="center"|<math>7~521~758</math>
 
 
|-
 
|-
!align="left"|Ratio of missing science samples vs samples stored
+
|align="left"|Ratio of missing science samples vs samples stored || 5&times;10<sup>-6</sup> || 7&times;10<sup>-6</sup>||7&times;10<sup>-6</sup>  
| align="center"|<math>5\times10^{-6}</math> || align="center"|<math>7\times10^{-6}</math>||align="center"|<math>7\times10^{-6}</math>  
 
 
|-
 
|-
 
|}
 
|}
 +
</center>
  
* <sup>(1)</sup> : mission periods are defined in [[HFIpreprocessingstatics|this page]].
+
* <sup>(1)</sup> Mission periods are defined on [[HFIpreprocessingstatics|this page]].
  
* <sup>(2)</sup> : science packets refer to the number of telemetry packets containing science data (ie: bolometer data and ''fine thermometer'' data) when the instrument is in observation mode (Application Program IDentifier = 1412). HSK packets refer to the number of HFI ''non essential'' housekeeping telemetry packets (Application Program IDentifier = 1410).
+
* <sup>(2)</sup> Science packets refer to the number of telemetry packets containing science data (i.e., bolometer data and ''fine thermometer'' data) when the instrument is in observation mode (Application Program IDentifier = 1412). HSK packets refer to the number of HFI ''non-essential'' housekeeping telemetry packets (Application Program IDentifier = 1410).
  
* <sup>(3)</sup> : all lost packets have been lost on board ; no HFI packet has been lost at ground segment level.
+
* <sup>(3)</sup> All lost packets have been lost on board; no HFI packet was lost at ground segment level.
  
* <sup>(4)</sup> : this amount of lost science samples is distributed as
+
* <sup>(4)</sup> This amount of lost science samples is distributed as:
**  0.3% are due to Single Event Unit
+
**  0.3% were due to a Single Event Unit;
** 18.5% have been lost during the 3 CDMU patch days and the consecutive clock resynchronisation
+
** 18.5% were lost during the three CDMU patch days and the consecutive clock resynchronization;
** 29.1% have been lost because of compression errors
+
** 29.1% were lost because of compression errors;
** 52.1% have been lost due to the EndOfSlew buffer overflow being triggered by solar flare events
+
** 52.1% were lost due to the EndOfSlew buffer overflow being triggered by solar flare events.
  
 
==Pointing data ==
 
==Pointing data ==
The pointing data are built by the MOC Flight Dynamics team. The pointing data are made available to the DPC via AHF files. See {{ICD|AHF|link=AHF description document}} and {{PLAOp|AHF|link=AHF files repository}}. All data contained in the AHF files are ingested in the HFI database. The present section describes the steps to produce the HFI pointing solution.
+
The pointing data are built by the MOC Flight Dynamics team. The pointing data are made available to the DPC via AHF files (see {{ICD|AHF|link=AHF description document}} and {{PLAOp|AHF|link=AHF files repository}}). All data contained in the AHF files are ingested into the HFI database. The present section describes the steps taken to produce the HFI pointing solution.
  
* During the stable pointing period (ie during the dwell), data sampling rate is given at 8 Hz while it is at 4 Hz during the satellite slews. Those data are thus interpolated to the bolometer sampling rate using a spherical linear interpolation as described in section 3.4 of of {{PEarly|6|page=6}}.
+
* During the stable pointing period (i.e., during the dwell), the data sampling rate is 8Hz, while it is 4Hz during the satellite slews. Those data are thus interpolated to the bolometer sampling rate using a spherical linear interpolation, as described in section 3.4 of of {{PlanckPapers|planck2011-1-7}}.
* The pointing solution is then amended from the wobble effect as delivered by MOC Flight Dynamics in the AHF files.  
+
* The pointing solution is then amended from the "wobble effect," as delivered by MOC Flight Dynamics in the AHF files.  
* A final correction is then applied based on the study of main planets and point sources seen by the bolometers.
+
* A final correction is then applied based on the study of the main planets and point sources seen by the bolometers.
 
   
 
   
 
 
''Note'': an
 
''Note'': an
'''HFI ring''' corresponds to each stable pointing period, when the spin axis is pointing towards an essentially fixed direction in the sky and the detectors repeatedly scan the same circle on the sky. More precisely the '''HFI ring ''' start time is defined as the ''time of the first thurster firing''. The end time of the HFI ring is the start time of the following ring.
+
"HFI ring" corresponds to each stable pointing period, when the spin axis is pointing towards an essentially fixed direction in the sky and the detectors repeatedly scan the same circle on the sky. More precisely the HFI ring start time is defined as the ''time of the first thruster firing''. The end time of the HFI ring is the start time of the following ring.
 
 
 
 
The plot below shows the evolution of the ring duration along the whole mission. It mainly reflects the scanning strategy and few ''long'' rings due to operational constraints or tests.
 
[[File:HFI_4_4_1_ringduration.png|500px|thumb|center|ring duration]]
 
  
 +
The plot below shows the evolution of the ring duration along the whole mission. It mainly reflects the scanning strategy and few ''long'' rings arising due to operational constraints or tests.
 +
[[File:HFI_4_4_1_ringduration.png|500px|thumb|center|Ring duration over the entire mission.]]
  
 
==Orbit data==
 
==Orbit data==
Satellite orbit velocity is built by the MOC Flight Dynamics team and made available to the HFI DPC via orbit files. See {{ICD|ODAT|link=Orbit description document}} and {{PLAOp|ORB|link= orbit files repository}}.
+
Satellite orbit velocity is built by the MOC Flight Dynamics team and made available to the HFI DPC via orbit files (see {{ICD|ODAT|link=Orbit description document}} and {{PLAOp|ORB|link= orbit files repository}}).
As those orbit files contain both effective and predictive data, they are regularly ingested updated in the HFI database.<br>
+
Since those orbit files contain both effective and predictive data, they are regularly ingested and updated in the HFI database.<br>
''Note: these same data are also ingested in parallel into the NASA JPL Horizons system (http://ssd.jpl.nasa.gov/) under ESA’s responsibility.''
+
''Note: these same data are also ingested in parallel into the [http://ssd.jpl.nasa.gov/|NASA JPL Horizons system] under ESA’s responsibility.''
  
The satellite orbit data preprocessing is the following.
+
The satellite orbit data preprocessing has the following steps.
* The sampling of the MOC provided orbit velocity data is appromximately 1 every 5 mn. These data are interpolated to the time of the middle of the HFI rings.
+
* The sampling of the MOC-provided orbit velocity data is approximately once every 5 min. These data are interpolated to the time of the middle of the HFI rings.
* The reference frame of the orbit data is translated from the MOC given Earth Mean Equator and Equinox J2000 (EME2000) reference frame to the ecliptic reference frame in cartesian coordinates.
+
* The reference frame of the orbit data is translated from the MOC-given Earth Mean Equator and Equinox J2000 (EME2000) reference frame to the Ecliptic reference frame in Cartesian coordinates.
* The earth velocity provided by the NASA JPL Horizons system (http://ssd.jpl.nasa.gov/) is interpolated to the time of the middle of the HFI rings. It is then added to the satellite velocity data.
+
* The Earth velocity provided by the NASA JPL Horizons system is interpolated to the time of the middle of the HFI rings. It is then added to the satellite velocity data.
  
 
The use of the satellite orbit velocity is three-fold:
 
The use of the satellite orbit velocity is three-fold:
# computation of the CMB orbital dipole (see [[Map-making | calibration section]]),
+
# computation of the CMB orbital dipole (see [[Map-making | calibration section]]);
# computation of the positions of solar system objects (see [[TOI_processing | data masking section]]),
+
# computation of the positions of solar system objects (see [[TOI_processing | data masking section]]);
# computation of the abberation correction.
+
# computation of the aberration correction.
  
 
==Time correlation data==
 
==Time correlation data==
The MOC is responsible for providing information about the relationship between the satellite On Board Time (OBT) and Coordinated Universal Time (UTC). This information comes via several measurements of OBT and UTC pairs each day, measured during the satellite ranging. See {{ICD|TCO|link=Time Correlation document}}.
+
The MOC is responsible for providing information about the relationship between the satellite On Board Time (OBT) and Coordinated Universal Time (UTC). This information comes via several measurements of OBT and UTC pairs each day, measured during the satellite ranging (see {{ICD|TCO|link=Time Correlation document}}).
  
The following plot shows the (OBT, UTC) data couples provided by MOC since Planck launch. The  x axis is in number of days since the 17th of May 2009 while the y axis is the difference of OBT and UTC in seconds. The very slow drift is approximately 0.05 seconds per day. The small wiggles are due to the satellite global temperature trends due to its orbit and distance to the sun.
+
The following plot shows the (OBT, UTC) data couples provided by MOC since Planck launch. The  <i>x</i>-axis is in number of days since the 17 May 2009, while, the <i>y</i>-axis is the difference of OBT and UTC in seconds. The very slow drift is approximately 0.05 seconds per day. The small wiggles come from the satellite global temperature trends, due to its orbit and distance to the Sun.
  
''Note: The isolated point on the 1st of June 2010 (just above 400 in abscisse) is not significant and due to a misconfiguration of the ground station parameters.''
+
''Note: The isolated point on 1 June 2010 (just above 400 in the abscissa) is not of any consequence, being due to a misconfiguration of the ground station parameters.''
  
 
[[File:HFI_4_4_1_OBTUTC.png|500px|thumb|center|OBT-UTC correlation data]]  
 
[[File:HFI_4_4_1_OBTUTC.png|500px|thumb|center|OBT-UTC correlation data]]  
  
An order 3 polynomial is then fitted to the data. As the whole HFI data management and processing uses OBT, the OBT-UTC information is only used when importing orbit data from Horizons.
+
A third-order polynomial is then fitted to the data. As the whole HFI data management and processing uses OBT, the OBT-UTC information is only used when importing orbit data from Horizons.
  
 
== References ==
 
== References ==
<biblio force=false>
+
<References />
#[[References]]
+
</biblio>
+
 
  
[[Category:HFI Data processing|001]]
+
[[Category:HFI data processing|001]]

Latest revision as of 07:23, 5 July 2018

Overview[edit]

In terms of data processing, the HFI ground segment handles two types of data, both made available via the MOC:

  • telemetry data transmitted from the satellite, coming from the different subsystems of the satellite service module, from the sorption cooler, and from the two instruments;
  • auxiliary data, produced by the MOC, with the only three products used by the HFI DPC being the pointing data, the orbit data, and the time correlation data.

All data are retrieved by the HFI Level 1 software and stored in the HFI database.

Telemetry data[edit]

The digitized data from the satellite are assembled on board in packets, according to the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations, and the ESA Packet Utilization Standard. The packets are dumped to the ground during the Daily Tranmission Control Period, consolidated by and stored at the MOC. Telemetry data contain the housekeeping data and the bolometer (i.e., science) data.

Housekeeping data[edit]

For several reasons (systems monitoring, potential impact of the environment, understanding of the bolometer data), the HFI Level 1 software gathers and stores in its database the satellite subsystems housekeeping parameters:

  • Command and Data Management System;
  • Attitude Control & Measurement Sub-system;
  • Thermal Control System;
  • Sorption Cooler System;
  • HFI housekeeping parameters.

The structure and the frequency of the packets built by these subsystems and the format of the housekeeping parameters are described in the Mission Information Bases (MIBs). The HFI L1 software uses these MIBs to extract the housekeeping parameters from the packets. Given the status of each subsystem, the parameters are gathered in the HFI database in groups. The HFI L1 software builds in each group a vector of time (usually named TIMESEC) and a single vector per housekeeping parameter.

Bolometer data[edit]

The HFI science data are retrieved and reconstructed as described in section 3.1 of Planck-Early-VI[1].

The first stage of HFI data processing is performed onboard in order to generate the telemetry. This is described in the data compression section.

On board, the signal from the 72 HFI channels is sampled at 180.4Hz by the Read-out Electronic Unit. 254 samples per channel are grouped into a compression slice. The Data Processing Unit then builds a set of several telemetry packets containing the compression slice data and adds to the first packet the start time of the compression slice. When receiving this set of telemetry packets, the L1 software extracts the 72 × 254 samples and computes the time of each sample based on the compression slice start time (digitized with 2-16 s = 15.26μs quantization steps) and a mean sample time between samples. For the nominal instrument configuration, the sample integration time is measured to be Tsamp = 5.54404ms.

The two plots below show the histogram of time differences between two successive samples for the full mission. Six sample intervals (> 6 ms) correspond to three occurrences of packets lost on board when the on-board software of the Command and Data Management Unit was patched (18, 19, and 20 of August 2009). The right plot is a blow-up of the left one. It shows the distribution around the mean value of 5.54ms and the 15μs quantization step.

HFI 4 4 1 deltatos.png
HFI 4 4 1 deltatoszoom.png

Transfer functions[edit]

In order to ease the reading of data in the HFI database, so called transfer functions are created. They allow software items to read data onto which functions are applied on the fly. A simple example of a transfer function is the conversion of thermometer data in analogue-to-digital Units (ADUs) to kelvins. Here we show the two transfer functions applied to the bolometer samples in the data processing pipelines.


From raw signal to non-demodulated signal in volts (transf1_nodemod):

[math]Signal\_{N\!D}\_V_{bc} = \frac{\frac{C_{bc}}{N_{\rm sample} - N_{\rm blanck,} bc}- O_{bc}}{F1_{bc}} \times \frac{G_{\rm amp, ETAL}}{G_{{\rm amp}, bc}} [/math].

The different symbols are described below.

  • bc refers to a bolometer usually labelled by its electronic belt and channel, e.g., bc=00 refers to the a part of the first 100GHz polarization sensitive bolometer.
  • Cbc is the bolometer sample in ADU.
  • Nsample is the number of samples in half a modulation period. This parameter is common to all bolometer channels and (although kept fixed during the whole mission) is read from the housekeeping parameter. Nsample=40.
  • Nblanck, bc is the number of samples suppressed at the beginning of each half-modulation period. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For bc=00, Nblanck, 00=0.
  • F1bc is a calibration factor. For bc=00, F100≈1.8×107.
  • Obc is close to 32768.
  • Gamp, ETAL is the gain amplifier measured during the calibration phase. For bc=00, Gamp,ETAL=1.
  • Gamp, bc is the current gain amplifier. Although kept fixed during the whole mission, it is read from the housekeeping parameter. For bc=00, Gamp,00=1.


From raw signal to demodulated signal in Volts (transf1):

the same formula as above is used, but with the demodulation and a 3-point filter computed as

[math]Signal\_V_{bc} [n] = - \frac{1}{4} (-1)^{{\rm parity}_{n-1}} Signal\_{N\!D}\_V_{bc} [n-1] + \frac{1}{2} (-1)^{{\rm parity}_{n}} Signal\_{N\!D}\_V_{bc} [n] - \frac{1}{4} (-1)^{{\rm parity}_{n+1}} Signal\_{N\!D}\_V_{bc} [n+1][/math],

where

  • n-1 and n+1 refer to the samples before and after the given n sample to demodulate,
  • "parity" is computed by the HFI L1 software.

Statistics on the telemetry data[edit]

The following table gives some statistics about the data handled at the pre-processing level.

Nominal mission(1) Full mission(1) From launch to the end of full mission(1)
Duration 473 days 884 days 974 days
Number of HFI packets generated onboard (HSK/science)(2) 19 668 436 / 376 294 615 37 762 493 / 704 852 262 41 689 4090 / 765 043 713
Number of HFI packets lost (3) (HSK/science) 2 / 20 2 / 20 2 / 20
Ratio of HFI lost packets to those generated on board (HSK/science) 1×10-7 / 5×10-8 5×10-8 / 3×10-8 5×10-8 / 3×10-8
Number of different housekeeping parameters stored in the database (HFI/SCS/sat) 3174 / 708 / 12390
Number of science samples stored in the database 530 632 594 653 991 929 524 565 1 090 125 748 960
Number of missing science samples 2 537 499 6 634 4914 7 521 758
Ratio of missing science samples vs samples stored 5×10-6 7×10-6 7×10-6
  • (1) Mission periods are defined on this page.
  • (2) Science packets refer to the number of telemetry packets containing science data (i.e., bolometer data and fine thermometer data) when the instrument is in observation mode (Application Program IDentifier = 1412). HSK packets refer to the number of HFI non-essential housekeeping telemetry packets (Application Program IDentifier = 1410).
  • (3) All lost packets have been lost on board; no HFI packet was lost at ground segment level.
  • (4) This amount of lost science samples is distributed as:
    • 0.3% were due to a Single Event Unit;
    • 18.5% were lost during the three CDMU patch days and the consecutive clock resynchronization;
    • 29.1% were lost because of compression errors;
    • 52.1% were lost due to the EndOfSlew buffer overflow being triggered by solar flare events.

Pointing data[edit]

The pointing data are built by the MOC Flight Dynamics team. The pointing data are made available to the DPC via AHF files (see AHF description document and AHF files repository). All data contained in the AHF files are ingested into the HFI database. The present section describes the steps taken to produce the HFI pointing solution.

  • During the stable pointing period (i.e., during the dwell), the data sampling rate is 8Hz, while it is 4Hz during the satellite slews. Those data are thus interpolated to the bolometer sampling rate using a spherical linear interpolation, as described in section 3.4 of of Planck-Early-VI[1].
  • The pointing solution is then amended from the "wobble effect," as delivered by MOC Flight Dynamics in the AHF files.
  • A final correction is then applied based on the study of the main planets and point sources seen by the bolometers.

Note: an "HFI ring" corresponds to each stable pointing period, when the spin axis is pointing towards an essentially fixed direction in the sky and the detectors repeatedly scan the same circle on the sky. More precisely the HFI ring start time is defined as the time of the first thruster firing. The end time of the HFI ring is the start time of the following ring.

The plot below shows the evolution of the ring duration along the whole mission. It mainly reflects the scanning strategy and few long rings arising due to operational constraints or tests.

Ring duration over the entire mission.

Orbit data[edit]

Satellite orbit velocity is built by the MOC Flight Dynamics team and made available to the HFI DPC via orbit files (see Orbit description document and orbit files repository). Since those orbit files contain both effective and predictive data, they are regularly ingested and updated in the HFI database.
Note: these same data are also ingested in parallel into the JPL Horizons system under ESA’s responsibility.

The satellite orbit data preprocessing has the following steps.

  • The sampling of the MOC-provided orbit velocity data is approximately once every 5 min. These data are interpolated to the time of the middle of the HFI rings.
  • The reference frame of the orbit data is translated from the MOC-given Earth Mean Equator and Equinox J2000 (EME2000) reference frame to the Ecliptic reference frame in Cartesian coordinates.
  • The Earth velocity provided by the NASA JPL Horizons system is interpolated to the time of the middle of the HFI rings. It is then added to the satellite velocity data.

The use of the satellite orbit velocity is three-fold:

  1. computation of the CMB orbital dipole (see calibration section);
  2. computation of the positions of solar system objects (see data masking section);
  3. computation of the aberration correction.

Time correlation data[edit]

The MOC is responsible for providing information about the relationship between the satellite On Board Time (OBT) and Coordinated Universal Time (UTC). This information comes via several measurements of OBT and UTC pairs each day, measured during the satellite ranging (see Time Correlation document).

The following plot shows the (OBT, UTC) data couples provided by MOC since Planck launch. The x-axis is in number of days since the 17 May 2009, while, the y-axis is the difference of OBT and UTC in seconds. The very slow drift is approximately 0.05 seconds per day. The small wiggles come from the satellite global temperature trends, due to its orbit and distance to the Sun.

Note: The isolated point on 1 June 2010 (just above 400 in the abscissa) is not of any consequence, being due to a misconfiguration of the ground station parameters.

OBT-UTC correlation data

A third-order polynomial is then fitted to the data. As the whole HFI data management and processing uses OBT, the OBT-UTC information is only used when importing orbit data from Horizons.

References[edit]

(Planck) High Frequency Instrument

[ESA's] Mission Operation Center [Darmstadt, Germany]

Data Processing Center

European Space Agency

House-Keeping data

Sorption Cooler Subsystem (Planck)

Command and Data Management Unit

Attitude History File

Cosmic Microwave background

On-Board Time

Universal Time Coordinate(d)