Difference between revisions of "Pre-processing LFI"
m |
|||
(20 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
+ | {{DISPLAYTITLE:Pre-processing}} | ||
+ | |||
== Overview == | == Overview == | ||
The first processing level of the LFI DPC is the so called Level 1. The source data of the Level 1 software includes: | The first processing level of the LFI DPC is the so called Level 1. The source data of the Level 1 software includes: | ||
Line 6: | Line 8: | ||
** The Attitude History File (AHF) | ** The Attitude History File (AHF) | ||
** Time correlation data (time correlation coefficients and time couples) | ** Time correlation data (time correlation coefficients and time couples) | ||
+ | ** Command History data | ||
** The Sorption cooler out of limit data | ** The Sorption cooler out of limit data | ||
− | Only a subset of the raw housekeeping telemetry packets is daily processed and converted into TOIs, i.e. those relevant to the LFI | + | Only a subset of the raw housekeeping telemetry packets is daily processed and converted into TOIs, i.e. those relevant to the LFI Daily Quality Report production and the estimation of the LFI instrument systematic effects. |
− | == | + | == Scientific telemetry == |
Each LFI radiometer provides two analog outputs, one for each amplifier | Each LFI radiometer provides two analog outputs, one for each amplifier | ||
chain. In a nominal configuration, each output yields a sequence of alternating <math>V_{sky}</math>, <math>V_{load}</math> signals at the frequency of the phase switch. By changing the phase switches configuration, the output can be a sequence of either <math>V_{sky}</math> or <math>V_{load}</math> signals. | chain. In a nominal configuration, each output yields a sequence of alternating <math>V_{sky}</math>, <math>V_{load}</math> signals at the frequency of the phase switch. By changing the phase switches configuration, the output can be a sequence of either <math>V_{sky}</math> or <math>V_{load}</math> signals. | ||
Line 40: | Line 43: | ||
The digitized scientific data is then processed by the Radiometer Electronics Box Assembly (REBA) which runs the LFI on-board software. For each LFI detector, the REBA processes the data in the form of time series which are split into telemetry packets. To satisfy the LFI assigned telemetry budget limit of 53.5 Kbps, the REBA implements 7 acquisition modes (processing types) which reduce the scientific data rate by applying a number of processing steps. The following figure illustrates the main steps of the on-board processing and the corresponding processing types (PTypes). | The digitized scientific data is then processed by the Radiometer Electronics Box Assembly (REBA) which runs the LFI on-board software. For each LFI detector, the REBA processes the data in the form of time series which are split into telemetry packets. To satisfy the LFI assigned telemetry budget limit of 53.5 Kbps, the REBA implements 7 acquisition modes (processing types) which reduce the scientific data rate by applying a number of processing steps. The following figure illustrates the main steps of the on-board processing and the corresponding processing types (PTypes). | ||
− | [[File:lfi_ptypes.png|center|thumb|500px|Schematic representation of the scientific on-board processing, processing parameters and processing types for the LFI. The diagram shows the sequence of operations leading to each processing type: coadding, mixing, requantization (Requant) and compression (CMP).]] | + | [[File:lfi_ptypes.png|center|thumb|500px|'''Schematic representation of the scientific on-board processing, processing parameters and processing types for the LFI. The diagram shows the sequence of operations leading to each processing type: coadding, mixing, requantization (Requant) and compression (CMP).''']] |
; ''PType 0'': in this mode the REBA just packs the raw data of the selected channel without any processing. | ; ''PType 0'': in this mode the REBA just packs the raw data of the selected channel without any processing. | ||
− | ; ''PType 1'': consecutive <math>S_{sky}</math> or <math>S_{load}</math> | + | ; ''PType 1'': consecutive <math>S_{sky}</math> or <math>S_{load}</math> samples are coadded and stored as unsigned integers of 32 bits. The number of consecutive samples to be coadded is specified by the <math>N_{aver}</math> parameter. |
; ''PType 2'': in this mode, two main processing steps are applied. First, pairs of averaged <math>S_{sky}</math> and <math>S_{load}</math> samples, respectively <math>\overline{S}_{sky}</math> and <math>\overline{S}_{load}</math>, are mixed by applying two different gain modulation factors, GMF1 and GMF2: | ; ''PType 2'': in this mode, two main processing steps are applied. First, pairs of averaged <math>S_{sky}</math> and <math>S_{load}</math> samples, respectively <math>\overline{S}_{sky}</math> and <math>\overline{S}_{load}</math>, are mixed by applying two different gain modulation factors, GMF1 and GMF2: | ||
:<math> | :<math> | ||
Line 68: | Line 71: | ||
Packets generated by the REBA follow the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations and the ESA Packet Utilization Standard (PUS). The packet structure for an LFI scientific telemetry packet is shown in the following figure. | Packets generated by the REBA follow the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations and the ESA Packet Utilization Standard (PUS). The packet structure for an LFI scientific telemetry packet is shown in the following figure. | ||
− | [[ | + | [[File: tm_packet_structure.png|500px|center|thumb|'''LFI scientific telemetry packet structure. The Packet Header, Data Field Header and Packet Error Control are specified in the PUS standard. The source data field contains a Structure ID (SID), to uniquely determine the format and layout of the field itself. It is followed by a tertiary header containing the detector id, the phase switches status and the REBA processing parameters used. The subsequent structure depends on the phase switches configuration and REBA processing type applied. This example shows a packet with PType 0 data and with the nominal phase switches configuration.''']] |
+ | |||
+ | == From packets to raw TOIs == | ||
+ | On a daily basis, the LFI Level 1 software pipeline retrieves the housekeeping and scientific telemetry packets dumped from the satellite on-board memory through the MOC Data Disposition System (DDS). The Level 1 software has to recover most accurately the values of the original (averaged) sky and load samples acquired on-board. Data acquired with PTypes 4, 5 and 6 is first uncompressed. The loss-less compression applied on-board is simply inverted and the number of samples obtained is checked with the value stored in the tertiary header. | ||
+ | |||
+ | The digitized data, processed by the REBA, are not in physical units but in ADU (Analog to Digital Unit). Conversion of <math>S_{sky}</math> and <math>S_{load}</math> in Volt requires the Data Source Address (DSA), i.e. the radiometer and detector from which the data are generated, the blanking time (indicized by the Blancking Time Index, BTI), the DGI and the DOI. The DSA and BTI values are recovered from the packet tertiary header, while the DGI and DOI values are recovered from the LFI HK telemetry. Hence, the value in Volt is obtained as: | ||
+ | |||
+ | |||
+ | :<math> | ||
+ | V_i = \dfrac{S_i \cdot q_{ADC} - Z_{DAE} - 2.5}{G_{DAE}} - O_{DAE} \approx \dfrac{S_i - \tilde{Z}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI})}{\tilde{G}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI})} - \tilde{O}_{DAE}(\text{DSA}, \text{BTI}, \text{DOI}) | ||
+ | </math> | ||
+ | |||
+ | |||
+ | where <math>\tilde{G}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI})</math>, <math>\tilde{O}_{DAE}(\text{DSA}, \text{BTI}, \text{DOI})</math> and <math>\tilde{Z}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI})</math> are look-up tables estimated during the LFI ground calibration campaign with: | ||
+ | |||
+ | * <math>\tilde{G}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI}) \approx \dfrac{G_{DAE}}{q_{ADC}} </math>, | ||
+ | |||
+ | * <math>\tilde{O}_{DAE}(\text{DSA}, \text{BTI}, \text{DOI}) \approx O_{DAE}</math>, | ||
+ | |||
+ | * <math>\tilde{Z}_{DAE}(\text{DSA}, \text{BTI}, \text{DGI}) \approx \dfrac{Z_{DAE} + 2.5}{q_{ADC}}</math>. | ||
+ | |||
+ | This conversion is the only processing required by PTypes 0 and 3 and it is the last step in the processing of all the other processing types. Since PType 1 data is just coadded on-board, the division by <math>N_{aver}</math> is performed by the Level 1 software. PType 2 and 5 data have to be dequantized by: | ||
+ | |||
+ | :<math> | ||
+ | P_i = \dfrac{Q_i}{q} - \text{Offset} | ||
+ | </math> | ||
+ | |||
+ | |||
+ | and then demixed to obtain <math>\overline{S}_{sky}</math> and <math>\overline{S}_{load}</math>: | ||
+ | |||
+ | :<math> | ||
+ | \begin{eqnarray} | ||
+ | \overline{S}_{sky} & = & \dfrac{P_2 \cdot \text{GMF1} - P_1 \cdot \text{GMF2}}{\text{GMF1} - \text{GMF2}} \\ | ||
+ | \overline{S}_{load} & = & \dfrac{P_2 - P_1}{\text{GMF1} - \text{GMF2}} | ||
+ | \end{eqnarray} | ||
+ | </math> | ||
+ | |||
+ | == On-board time reconstruction == | ||
+ | The On-Board Time (OBT) reconstruction for scientific data has to take into account the [[LFI design, qualification, and performance#Radiometer Chain Assembly (RCA)|phase switch]] status and the processing type applied on-board. If the phase switch is off, it means that the packet contains consecutive values of either sky or load samples, and the sampling frequency, <math>f_{sampling}</math>, is 8192 Hz. Denoting with <math>i \ge 0</math> the sample index within the packet, for PType 0 and 4 we have that: | ||
+ | |||
+ | |||
+ | :<math> | ||
+ | t^{obt}_{i} = t^{obt}_{0} + i\dfrac{1}{f_{sampling}}, | ||
+ | </math> | ||
+ | |||
+ | |||
+ | where <math>t^{obt}_{0}</math> is the on-board time of the packet (<math>t^{obt}_{pkt}</math>) and <math>i = 0</math> denotes the first sample in the packet. If the switching status is on, either consecutive pairs of (sky, load) samples or (load, sky) samples are stored in the packets. Hence, consecutive couples of samples have the same time stamp and <math>f_{sampling}</math> = 4096 Hz. | ||
+ | |||
+ | For averaged data (PTypes 1, 2, 3, 5, 6), the first sample of a scientific packet is the sum (mean) of <math>N_{aver}</math> samples, and the packet time, <math>t^{obt}_{pkt}</math>, is the time of the first of the <math>N_{aver}</math> samples. In this case, <math>t^{obt}_{0}</math> is computed as: | ||
+ | |||
+ | |||
+ | :<math> | ||
+ | t^{obt}_{0} = t^{obt}_{pkt} + \dfrac{N_{aver}-1}{2} \dfrac{1}{f_{sampling}} | ||
+ | </math> | ||
+ | |||
+ | and | ||
+ | |||
+ | :<math> | ||
+ | t^{obt}_{i} = t^{obt}_{0} + i\dfrac{N_{aver}}{f_{sampling}}, \text{for} \ \ i \ge 1 | ||
+ | </math> | ||
+ | |||
+ | == Housekeeping telemetry handling == | ||
+ | The structure of telecommand and housekeeping telemetry packets of the entire satellite is defined by the Mission Information Base (MIB), a database formed by a set of ASCII tables formatted according to the ESA Mission Control System interface control documents. The MIB information includes the type and structure of the telemetry packets, the location, type and format of the monitoring parameters within the packets, the calibration curves to convert each parameter raw value into an engineering value, the out of limit values to be checked for each parameter. | ||
+ | |||
+ | The Level1 software uses the MIB information to group the housekeeping packets according to their type (PUS service type, sub-sytem, periodicity). A subset of the housekeeping packets that are relevant for the daily instrument quality verification and the scientific data analysis are further processed: samples of each parameter are extracted, grouped into timelines and saved as TOIs. Each TOI contains, for each parameter sample, the on-board time, the UTC time, the parameter raw and engineering value, flags reporting some quality measures (time quality, out of limit checking result). | ||
+ | |||
+ | |||
+ | == Auxiliary data handling == | ||
+ | The MOC Flight Dynamics team daily provides the Attitude History information as an ASCII file (AHF), automatically delivered through the Planck File Transfer System. For stable pointing periods, the AHF provides quaternions describing, at given on-board times, the orientation of the Planck body reference frame with respect to the Ecliptic inertial reference system, and additional information such as wobble angles, spin phase angle and rate. An AHF file contains also different type of records: high frequency data records containing the raw attitude data; spin period frequency records, that are derived from the high frequency records by averaging data over a complete spin period; observation frequency records, where data is averaged over a complete observation period. The Level1 software simply reformat all data contained in the AHF, ingesting it in the LFI Level1 data management system. | ||
+ | |||
+ | The MOC is also responsible of computing the correlation between the On-Board Time (OBT) and the ground Coordinated Universal Time (UTC) and providing the UTC time of each telemetry packet. Time couples (OBT, UTC), generated by processing the Standard Time Source packets received from spacecraft, and the Time Correlation Coefficients, computed by a linear fit of the time couples, are also provided by MOC as auxiliary data. The Level1 software uses the time correlation coefficients to check the UTC time provided by MOC. Moreover, the time couples are used to recompute the UTC time of each packet with a variation of the MOC fitting procedure, in order to reduce the time correlation variance. | ||
+ | [[Category:LFI data processing|001]] |
Latest revision as of 20:58, 17 March 2013
Contents
Overview[edit]
The first processing level of the LFI DPC is the so called Level 1. The source data of the Level 1 software includes:
- raw housekeeping telemetry packets retrieved from different satellite subsystems: the LFI instrument, the Sorption cooler, the HFI instrument and the Central Data Management Unit (CDMU).
- the LFI raw scientific telemetry
- Additional auxiliary data provided by the MOC and the Flight dynamics:
- The Attitude History File (AHF)
- Time correlation data (time correlation coefficients and time couples)
- Command History data
- The Sorption cooler out of limit data
Only a subset of the raw housekeeping telemetry packets is daily processed and converted into TOIs, i.e. those relevant to the LFI Daily Quality Report production and the estimation of the LFI instrument systematic effects.
Scientific telemetry[edit]
Each LFI radiometer provides two analog outputs, one for each amplifier chain. In a nominal configuration, each output yields a sequence of alternating , signals at the frequency of the phase switch. By changing the phase switches configuration, the output can be a sequence of either or signals.
The conversion from analog to digital form of each radiometer output is performed by a 14 bits Analog-to-Digital Converter (ADC) in the Data Acquisition Electronics unit (DAE). The DAE transforms the signal in the range [-2.5 V, +2.5 V]: first it applies a tunable offset, , then it amplifies the signal with a tunable gain, , in order to make full use of the resolution of the ADC, and finally the signal is integrated. To eliminate phase switch raise transients, the integration takes into account a blanking time, i.e. a blind time in the integrator where data are not considered. The default value of the blanking time is 7.5 . Both the , the and the blanking time are parameters set through the LFI on-board software. The equation applied to transform a given input signal into an output is:
with , is one of 255 possible offset steps from +0 up to +2.5 V and where is a small offset introduced by the DAE when applying a selected gain. The values of and are set by
sending, through specific telecommands, the DAE Gain Index (DGI) and the DAE
Offset Index (DOI) associated to the desired values.
The ADC quantizes the uniformly in the range , so that the quantization step is mV. The quantization formula is
and the output is stored as an unsigned integer of 16 bits.
The digitized scientific data is then processed by the Radiometer Electronics Box Assembly (REBA) which runs the LFI on-board software. For each LFI detector, the REBA processes the data in the form of time series which are split into telemetry packets. To satisfy the LFI assigned telemetry budget limit of 53.5 Kbps, the REBA implements 7 acquisition modes (processing types) which reduce the scientific data rate by applying a number of processing steps. The following figure illustrates the main steps of the on-board processing and the corresponding processing types (PTypes).
- PType 0
- in this mode the REBA just packs the raw data of the selected channel without any processing.
- PType 1
- consecutive or samples are coadded and stored as unsigned integers of 32 bits. The number of consecutive samples to be coadded is specified by the parameter.
- PType 2
- in this mode, two main processing steps are applied. First, pairs of averaged and samples, respectively and , are mixed by applying two different gain modulation factors, GMF1 and GMF2:
- The operations are performed as floating point operations. Then the two values obtained are requantized, converting them into two 16-bit signed integers:
- PType 3
- with respect to PType 2, in this mode only a single gain modulation factor is used, GMF1, obtaining:
- and analogously to PType 2, the value is requantized obtaining a 16-bit signed integer.
- PTypes 4, 5, 6
- with the processing types PType 4, PType 5 and PType 6, the REBA performs a loss-less adaptive arithmetic compression of the data obtained respectively with the processing types PType 0, PType 2 and PType 3. The compressor takes couples of 16 bit numbers and stores them in the output stream up to the complete filling of the data segment for the packet in process.
A set of REBA processing parameters — , GMF1, GMF2, q and Offset — is selected for each of the 44 LFI channels.They are also included in a teartiary header of each scientific telemetry packet sent to ground. The REBA can acquire data from a channel in two modes at the same time. This capability is used to verify the effect of a certain processing type on the data quality. So, in nominal conditions, the LFI instrument uses PType 5 for all its 44 detectors and every 15 minutes a single detector, in turn, is also processed with PType 1, in order to periodically check the gain modulation factors and the second quantization parameters. The other processing types are mainly used for diagnostic, testing or contingency purposes.
Packets generated by the REBA follow the ESA Packet Telemetry Standard and Packet Telecommand Standard, the CCSDS Packet Telemetry recommendations and the ESA Packet Utilization Standard (PUS). The packet structure for an LFI scientific telemetry packet is shown in the following figure.
From packets to raw TOIs[edit]
On a daily basis, the LFI Level 1 software pipeline retrieves the housekeeping and scientific telemetry packets dumped from the satellite on-board memory through the MOC Data Disposition System (DDS). The Level 1 software has to recover most accurately the values of the original (averaged) sky and load samples acquired on-board. Data acquired with PTypes 4, 5 and 6 is first uncompressed. The loss-less compression applied on-board is simply inverted and the number of samples obtained is checked with the value stored in the tertiary header.
The digitized data, processed by the REBA, are not in physical units but in ADU (Analog to Digital Unit). Conversion of and in Volt requires the Data Source Address (DSA), i.e. the radiometer and detector from which the data are generated, the blanking time (indicized by the Blancking Time Index, BTI), the DGI and the DOI. The DSA and BTI values are recovered from the packet tertiary header, while the DGI and DOI values are recovered from the LFI HK telemetry. Hence, the value in Volt is obtained as:
where , and are look-up tables estimated during the LFI ground calibration campaign with:
- ,
- ,
- .
This conversion is the only processing required by PTypes 0 and 3 and it is the last step in the processing of all the other processing types. Since PType 1 data is just coadded on-board, the division by
is performed by the Level 1 software. PType 2 and 5 data have to be dequantized by:
and then demixed to obtain and :
On-board time reconstruction[edit]
The On-Board Time (OBT) reconstruction for scientific data has to take into account the phase switch status and the processing type applied on-board. If the phase switch is off, it means that the packet contains consecutive values of either sky or load samples, and the sampling frequency, , is 8192 Hz. Denoting with the sample index within the packet, for PType 0 and 4 we have that:
where is the on-board time of the packet ( ) and denotes the first sample in the packet. If the switching status is on, either consecutive pairs of (sky, load) samples or (load, sky) samples are stored in the packets. Hence, consecutive couples of samples have the same time stamp and = 4096 Hz.
For averaged data (PTypes 1, 2, 3, 5, 6), the first sample of a scientific packet is the sum (mean) of
samples, and the packet time, , is the time of the first of the samples. In this case, is computed as:and
Housekeeping telemetry handling[edit]
The structure of telecommand and housekeeping telemetry packets of the entire satellite is defined by the Mission Information Base (MIB), a database formed by a set of ASCII tables formatted according to the ESA Mission Control System interface control documents. The MIB information includes the type and structure of the telemetry packets, the location, type and format of the monitoring parameters within the packets, the calibration curves to convert each parameter raw value into an engineering value, the out of limit values to be checked for each parameter.
The Level1 software uses the MIB information to group the housekeeping packets according to their type (PUS service type, sub-sytem, periodicity). A subset of the housekeeping packets that are relevant for the daily instrument quality verification and the scientific data analysis are further processed: samples of each parameter are extracted, grouped into timelines and saved as TOIs. Each TOI contains, for each parameter sample, the on-board time, the UTC time, the parameter raw and engineering value, flags reporting some quality measures (time quality, out of limit checking result).
Auxiliary data handling[edit]
The MOC Flight Dynamics team daily provides the Attitude History information as an ASCII file (AHF), automatically delivered through the Planck File Transfer System. For stable pointing periods, the AHF provides quaternions describing, at given on-board times, the orientation of the Planck body reference frame with respect to the Ecliptic inertial reference system, and additional information such as wobble angles, spin phase angle and rate. An AHF file contains also different type of records: high frequency data records containing the raw attitude data; spin period frequency records, that are derived from the high frequency records by averaging data over a complete spin period; observation frequency records, where data is averaged over a complete observation period. The Level1 software simply reformat all data contained in the AHF, ingesting it in the LFI Level1 data management system.
The MOC is also responsible of computing the correlation between the On-Board Time (OBT) and the ground Coordinated Universal Time (UTC) and providing the UTC time of each telemetry packet. Time couples (OBT, UTC), generated by processing the Standard Time Source packets received from spacecraft, and the Time Correlation Coefficients, computed by a linear fit of the time couples, are also provided by MOC as auxiliary data. The Level1 software uses the time correlation coefficients to check the UTC time provided by MOC. Moreover, the time couples are used to recompute the UTC time of each packet with a variation of the MOC fitting procedure, in order to reduce the time correlation variance.
(Planck) Low Frequency Instrument
Data Processing Center
(Planck) High Frequency Instrument
Command and Data Management Unit
[ESA's] Mission Operation Center [Darmstadt, Germany]
Attitude History File
analog to digital converter
LFI Data Acquisition Electronics
LFI Radiometer Electronics Box Assembly
European Space Agency
Packet Utilisation Standard
MOC's Data Distribution System
House Keeping
On-Board Time
Universal Time Coordinate(d)