Timelines

There will be no timelines in the 1st data release …

Semi-raw signal timelines

HFI timelines

Originally defined to be the raw (ADU) data demodulated and converted to engineering units (V) via the HFI transfer functions (includes 3-pt filtering?). But (1) this is not the same demodulation method as used in the TOI prod pipeline, and (2) the ADC correction (or at least one method thereof) implies a modification of the raw data before demodulation. Thus what the HFI will deliver is not yet decided

LFI timelines

TBW

Cleaned and calibrated signal timelines

HFI processing

The calibrated timelines are produced by the TOIprod pipeline (link to wiki; link to paper) which in brief performs the following operations:

demodulation

this is performed around a variable level which is determined from the valid input data (a validity flat from a previous version of the processing is determined for this purpose), and the data are converted to engineering units (V) using known conversion coefficients.

despiking

using the demodulated data converted to V (by the transfer function) the glitches are identified and fitted with templates. A glitch flag is produced that identifies the strongest part of the glitches, and a timeline of glitch tails is produced from the template fits, and subtracted from the demodulated timeline from step 1. Finally, the flagged ranges are replaced with data from TBC (is this done here, if not where?)

dark template removal

TBW

conversion to absorbed power

the timeline is converted to watts of absorbed power using the bolometer function. This includes a non-linearity correction

removal of the 4K cooler lines

TBW

at this point a timeline in W and a glitch flag are written to disk

deconvolution by the time transfer function

TBW

jump correction

removes some (relatively rare) jumps in the signal baseline

at this point a tau-deconvolved and jump-corrected timeline and its associated glitch flag are written to disk. These timelines are later used in the flux calibration and mapmaking stages

Note that these timelines contain the full sky signal, i.e. including the solar and orbital dipoles and the Zodiacal light. The dipoles are necessary for the flux calibration and are removed by the pipeline at the mapmaking stage.

Need to discuss dipoles removal and conversion to astrophysical units at the export stage

Pointing timelines

The pointing is determined starting from the AHF produced by MOC, which gives the direction and orientation of the LOS of a fiducial position in the focal plane at frequencies of 8Hz during stable pointing and 4 Hz during maneuvers (TBC for details, reference). This is interpolated to the times of data observation (ref to method), corrected for the wobble and other time-dependent offsets determined from the observed positions of a large number of sources around the sky, and finally converted to the LOS of each detector using the quaternions in the IMO (which are determined from observations of bright planets - see the Focal plane reconstruction pipeline).

Details of delivery are TBD

FITS file structure

The FITS files will begin with a minimal primary header that will be followed by several BINTABLE extensions: the first one for the OBT and its (global) flag, the following ones for each detector signal data and its (local) flag.

Filenames will be of the form {H|L}FI_HscN-DDDD-R-yyymmdd.fits, where DDDD is the OD number, sc indicates science data, and N=1-5 for the 5 HFI files, and R indicates it is reduced (i.e., cleaned and calibrated) data.

NB. this is the current name used in DPC exchanges, perhaps it should be changed … eg. replace the date with the release number. Will also indicate that they are calibrated in astrophysical units and will have the dipoles removed, thus they are for delivery to the external community.

Header keywords

An example is given below.

= EXTENSION 0:

SIMPLE  =                    T / file does conform to FITS standard             
BITPIX  =                   32 / number of bits per data pixel                  
NAXIS   =                    0 / number of data axes                            
EXTEND  =                    T / FITS dataset may contain extensions            
COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H 
EFDD_VER= '99.9    '                                                            
ICD_VER = '99.9    '                                                            
IMO     = 'IMO_2_62'                                                            
DATE    = '2012-09-04T13:45:59' / file creation date (YYYY-MM-DDThh:mm:ss UT)   
END                                                                             

where EFDD_VER and ICD_VER are the versions of the respective documents in which the structure is defined (dummy values in the example)

EXTENSION 1: OBT

XTENSION= 'BINTABLE'           / binary table extension                         
BITPIX  =                    8 / 8-bit bytes                                    
NAXIS   =                    2 / 2-dimensional binary table                     
NAXIS1  =                    9 / width of table in bytes                        
NAXIS2  =             15573106 / number of rows in table                        
PCOUNT  =                    0 / size of special data area                      
GCOUNT  =                    1 / one data group (required keyword)              
TFIELDS =                    2 / number of fields in each row                   
TTYPE1  = 'OBT     '           / label for field   1                            
TFORM1  = '1K      '           / data format of field: 8-byte INTEGER           
TUNIT1  = '2**(-16)sec'        / physical unit of field                         
TTYPE2  = 'Flag    '           / label for field   2                            
TFORM2  = '1B      '           / data format of field: BYTE                     
TUNIT2  = 'N/A     '           / physical unit of field                         
EXTNAME = 'OBT     '           / name of this binary table extension            
TIMEZERO= '1660237181'                                                          
END                                                                             
  

where TIMEZERO is the value of the first timesample in the units of TUNIT1

EXTENSION 2: 14_545_1_LFER4_JC_v51

XTENSION= 'BINTABLE'           / binary table extension                         
BITPIX  =                    8 / 8-bit bytes                                    
NAXIS   =                    2 / 2-dimensional binary table                     
NAXIS1  =                    5 / width of table in bytes                        
NAXIS2  =             15573106 / number of rows in table                        
PCOUNT  =                    0 / size of special data area                      
GCOUNT  =                    1 / one data group (required keyword)              
TFIELDS =                    2 / number of fields in each row                   
TTYPE1  = 'Signal  '           / label for field   1                            
TFORM1  = '1E      '           / data format of field: 4-byte REAL              
TUNIT1  = 'KCMB    '           / physical unit of field                         
TTYPE2  = 'Flag    '           / label for field   2                            
TFORM2  = '1B      '           / data format of field: BYTE                     
TUNIT2  = 'N/A     '           / physical unit of field                         
EXTNAME = '14_545_1_LFER4_JC_v51' / name of this binary table extension         
TIMEZERO= '1660237181'                                                          
CHANNEL = '545-1   '                                                            
END                                                                             

Note that NAXIS must be the same for all extensions.

NB. may want to change EXTNAME to be just the detector name; the rest was for internal purposes only