TOI processing LFI

Overview

The LFI Level2 Pipeline analyzes each horn of the instrument separately, one pointing period at time and store results in object the length of an OD. Each diode of the horn is corrected from systematic, differentiated and then combined with its couple in the same radiometer. The horn is then calibrated and the photometric calibration is applied.

Pre-processing

Before the run of the Level2 pipeline and to improve the analysis the Mission information and data sampling division are stored in the database.

The Mission information is a set of object, one for each Operational Day (OD, as defined in MISSING), in which are stored Pointing Period data: DPC pointing ID (where 1 is the first pointing of nominal mission), PSO pointing ID, start OBT of the pointing maneuver, start OBT of the stable pointing, end OBT of the pointing, spin axis ecliptic longitude and latitude.

The sampling information is a set of object, one for each LFI frequency, in which are stored for each pointing ID: start OBT of the pointing maneuver, start OBT of the stable pointing, end OBT of the pointing, number of samples of the pointing, number of stable samples of the pointing, start sample of the stable pointing and sample number from the start of the nominal mission. The sample number for each pointing is given by intersecting OBT of all the radiometers at the same frequency.

ADC Correction

More on P02 and P02a.

Evaluation

Bob sent me the email. Copy the page: http://belzebu.lambrate.inaf.it/lfiWiki/bin/view/LFI/AdcCharacterisation

Application

For each of the 44 LFI diodes there is the corresponding object in the Database. Each object contains 4 columns: the input voltages coming from the sky channel and the corresponding linearized output, the input voltages coming from the reference channel and the corresponding linearized output.

Data loaded by the module are used to initialize two different interpolator using CSPLINE and the functions from gsl (GNU Scientific Libraries) libraries. The interpolators are then used to correct each sample.

Spikes Removal

Some of the LFI receivers exhibit a small artifact, visible in the power spectra. The effect is a set of spikes at 1 Hz and harmonics. More information can be found in P02 and P02a.

Modelization

The method to estimate the 1 Hz signal is to make use of linear filtering of a template. The first step is dividing the data into bins: this is done by taking fractional part of the sample time in seconds (using on-board time) and placing the time sample into a bin depending upon the particular fraction of a second in which it lays. The number of bins is computed using:

[math] nbins = fsamp * template\_resolution[/math]

where fsamp is the sampling frequency, while template resolution is 136 at 70 GHz, 80 at 44 GHz and 56 at 30 GHz. Then the bins vector is initialized with time intervals.

We can write the process adding an index to the time sample: lower index denotes the particular time sample, while the upper index labels the bin into which the sample falls. The linear filter can be written as:

[math] s(t_{i}^{j}) = a_j \left(1- \Delta x (t_{i}^{j}) \right) + a_{j+1} \Delta x (t_{i}^{j})[/math]

Here [math] \Delta x (t_{i}^{j})[/math] is the filter weight which is determined by where within the bin sample lies. If we use [math] t^j [/math] with only an upper index to denote the start of each bin, then we can write the filter wright as follows:

[math] \Delta x (t_{i}^{j}) = {{{t_i^j - t^j} \over {t^{j+1} - t^j}}} [/math]

In other words, the filter weight is the time sample value minus the start of the bin divided by the width of the bin.

We must estimate the parameters [math] a_j [/math] from the data. With the assumption that the instrument has stable noise properties, we can use a least square algorithm to estimate the bin values:

[math] {\partial \over \partial a_k} \sum_{i,j} \left( s(t_i^j) – d_i^j \right)^2 = 0 [/math]

This can be represented in matrix equation:

[math] M_{jk}a_k = b_j [/math]

with the following definitions:

[math] M_{k,k-1} = \sum_i (1 - \Delta x (t_i^{k-1})) \Delta x (t_i^{k-1}) [/math]

[math] M_{k,k} = \sum_i (1 - \Delta x (t_i^k))^2 \Delta x (t_i^{k-1})^2 [/math]

[math] M_{k,k+1} = \sum_i (1 - \Delta x (t_i^k)) \Delta x (t_i^k) [/math]

[math] M_{k,k+n} (|n| \gt 1) = 0 [/math]

[math] b_j = \sum_i d_i^k (1- \Delta x (t_k^i)) + d_i^{k-1}\Delta x (t_i^{k-1}) [/math]

With that definitions we have to make use of periodic boundary conditions to obtain the correct results, such that if [math] k = 0 [/math], [math] k-1 = n-1 [/math] and [math] k = n-1 [/math], [math] k+1 = 0 [/math]. Once this is done, we have a symmetric tridiagonal matrix with additional values at the upper right and lower left corners of the matrix. The matrix is solved with LU decomposition. In order to be certain of the numerical accuracy of the result, we can perform a simple iteration. The solving of the linear system and the iterative improvement of the solution are implemented as suggested in Numerical Recipes.

Application

For each of the 44 LFI diodes there is the corresponding object in the Database. Because of the amplitude of the spikes we choose to apply correction only on the 44 GHz radiometers. Each object contains 3 columns: the bins start time vector, the sky amplitudes and the reference amplitudes.

For each sample the value to be subtracted is computed using:

[math] V = skyAmp_k (1 - \Delta x (t_k)) + skyAmp_{k+1} \Delta x (t_k) [/math]

where k is the index of the bins at a given time.

Gaps Filling

During the mission some of the data packets were lost (see P02). Moreover in two different and very peculiar situation LFI was shutdown and restart, giving inconsistencies in data sampling. All of those data aren't used for scientific purpose but to avoid discrepancies in data analysis all of the radiometers at the same frequency must have the same number of samples.

To accomplish this purpose the length of the data stream to be reduced in a specific pointing period is compared with the data stored in the sample information object. If the length is not the same the OBT vector is filled with missing samples, the data vector is filled with zeros and in the flag column the bit for gap is raised.

Gain Modulation Factor

As result of the pseudo-correlation design of the LFI radiometers the [math] 1/f [/math] noise can be dramatically reduced differentiating the [math] V_sky [/math] and [math] V_load [/math] outputs. The two streams are slightly unbalanced, as one looks at the 2.7 K sky and the other looks at the ~4.5 K reference load. To force the mean of the difference to zero, the load signal is multiplied by the Gain Modulation Factor (R). For each pointing period the factor is computed using:

[math] R = \lt V_{sky}\gt \over \lt V_{load}\gt [/math]

Then the data are differentiated using:

[math] DD = V_{sky} – R V_{load} [/math]

At the same time the maneuver flag bit is raised using the information stored in the sampling information object to remove those data from the next step of the Pipeline.

The R values are stored in the database. At the same time the mean values are stored in order to be used in other steps of the analysis.

Diode Combination

The combination of the diodes has to take in account effects of differences in noise between the two diodes. To combine the diodes we assign relative weights to the uncalibrated diode streams based on their calibrated noise.

Evaluation

From first order calibration we compute an absolute gain [math] G_0 [/math] and [math] G_1 [/math], subtract an estimated sky and calculate the calibrated white noise [math] \sigma_0 [/math] and [math] \sigma_1 [/math], for the couple of diodes. The weights for the two diodes ([math] i [/math] = 0 or 1) are:

[math] W_i = {\sigma_i^2 \over G_{01}} {1 \over {\sigma_0^2 + \sigma_1^2}} [/math]

where the weighted calibration constant is given by:

[math] G_{01} = {1 \over {\sigma_0^2 + \sigma_1^2}} [G_0 \sigma_1^2 + G_1 \sigma_0^2] [/math]

The weights are fixed to a single value per diode for the entire dataset.

Detector ID	Weight
LFI18M-00	0.567304963
LFI18M-01	0.432695037
LFI18S-10	0.387168785
LFI18S-11	0.612831215
LFI19M-00	0.502457723
LFI19M-01	0.497542277
LFI19S-10	0.55143474
LFI19S-11	0.44856526
LFI20M-00	0.523020094
LFI20M-01	0.476979906
LFI20S-10	0.476730576
LFI20S-11	0.523269424
LFI21M-00	0.500324722
LFI21M-01	0.499675278
LFI21S-10	0.563712153
LFI21S-11	0.436287847
LFI22M-00	0.536283158
LFI22M-01	0.463716842
LFI22S-10	0.553913461
LFI22S-11	0.446086539
LFI23M-00	0.508036034
LFI23M-01	0.491963966
LFI23S-10	0.36160661
LFI23S-11	0.63839339
LFI24M-00	0.602269189
LFI24M-01	0.397730811
LFI24S-10	0.456037835
LFI24S-11	0.543962165
LFI25M-00	0.482050606
LFI25M-01	0.517949394
LFI25S-10	0.369618239
LFI25S-11	0.630381761
LFI26M-00	0.593126369
LFI26M-01	0.406873631
LFI26S-10	0.424268188
LFI26S-11	0.575731812
LFI27M-00	0.519877701
LFI27M-01	0.480122299
LFI27S-10	0.484831449
LFI27S-11	0.515168551
LFI28M-00	0.553227696
LFI28M-01	0.446772304
LFI28S-10	0.467677355
LFI28S-11	0.532322645

Planet Flagging

Why we flag planets.

Looking for Planets in data

Michele's code. Under review.

Application

The OBT vector found by the search are saved in a set of object, one for each horn. In Level2 Pipeline those OBTs are compared with the OBT vector of the data to raise planet bit where needed.