Difference between revisions of "BeyondPLANCK"

From Planck Legacy Archive Wiki
Jump to: navigation, search
Line 1: Line 1:
 
== BeyondPlanck ==
 
== BeyondPlanck ==
  
'''Disclaimer: the following data sets have been delivered and ingested in the PLA as agreed between ESA and the BeyondPlanck project reprocessing Planck data. Please note that these products have not been produced by the Planck Collaboration and we do not provide any kind of support for the content of these products. For questions about these products please contact the BeyondPlanck project directly.'''
+
'''Disclaimer''': the following data sets have been delivered and ingested in the PLA as agreed between ESA and the BeyondPlanck project reprocessing Planck data. Please note that these products have not been produced by the Planck Collaboration and we do not provide any kind of support for the content of these products. For questions about these products please contact the BeyondPlanck project directly.
  
 
=== Overview and background ===
 
=== Overview and background ===

Revision as of 08:27, 14 February 2023

BeyondPlanck[edit]

Disclaimer: the following data sets have been delivered and ingested in the PLA as agreed between ESA and the BeyondPlanck project reprocessing Planck data. Please note that these products have not been produced by the Planck Collaboration and we do not provide any kind of support for the content of these products. For questions about these products please contact the BeyondPlanck project directly.

Overview and background[edit]

As of 2020 and into the foreseeable future, ESA's Planck satellite measurements represent the state-of-art in terms of full-sky observations of the microwave sky between 30 and 857GHz, and the work by the Planck collaboration in terms of processing and interpreting these observations are summarized in a series of 150 papers. Despite these massive efforts, several known outstanding issues regarding the final Planck maps remained unresolved at the end of the funded analysis phase, and several external initiatives were started to address these. One of these was BeyondPlanck, which aimed to re-process the Planck LFI data within a self-consistent end-to-end Bayesian framework, in which both instrumental and astrophysical parameters are fitted jointly. This work started in 2018, and was successfully concluded in 2022. The final results from this project are described in a series of 17 papers, and the products made available through the Planck Legacy Archive. A long-term goal is to apply these techniques to a wide range of state-of-the-art experiments, and this work is organized within the Cosmoglobe project.

Data model[edit]

The main novel feature of the BeyondPlanck processing is a single joint parametric data model that accounts for both astrophysical and instrumental parameters on the form:

BP model.png

where

  • t denotes time sample and j denotes LFI radiometer index
  • g denotes the instrumental gain
  • P is the pointing matrix
  • M denotes the component-frequency mixing matrix, which depends on both astrophysical foreground spectral parameters, beta, and the instrument bandpass
  • a^c denotes the spatial amplitude map of component c
  • B denotes beam convolution (either with the symmetric main beam response, the asymmetric far-sidelobe beam response, or full 4pi response)
  • s^orb denotes the orbital CMB dipole
  • s^fsl denotes the sky signal observed through the far sidelobes
  • s^1Hz denotes electronic 1Hz spike contamination
  • n^corr denotes instrumental correlated noise
  • n^w denotes instrumental white noise

Explicitly, the sum over components reads as follows (neglecting for notational simplicity bandpass integration; this is accounted for in the actual processing)

BP astmodel.png

In addition to these explicit parameters, the model also includes hyper-parameters for some of the stochastic fields, perhaps most notably the CMB power spectrum, C_l, which describes the variance of the CMB field, and the correlated noise power spectral density, N_corr(f), as a function of temporal frequency.

For further information regarding this data model, see BeyondPlanck overview paper and references therein.

Data sets[edit]

The above data model is fitted using the following combination of datasets:

Posterior distribution and Gibbs sampling[edit]

Let omega be the set of all free parameters in the above data model. The BeyondPlanck data model is then fitted to the listed data sets by mapping out the full joint posterior distribution by Monte Carlo sampling,

BP posterior.png

where the likelihood is defined by the white noise distribution

BP likelihood.png

The prior, P(omega), is summarized in the BeyondPlanck overview paper, and consists of a mixture of informative and algorithmic priors.

In practice, the posterior mapping is performed using Gibbs sampling with the following Gibbs chain,

BP gibbs.png

Four independent chains are run for each 1000 iterations, and the first 200 samples are excluded as burn-in, leaving a total of 3200 samples for final analysis. All products listed below are derived from those post-burn-in 3200 samples.

Products[edit]

The BeyondPlanck products are available both through the Planck Legacy Archive and through the Cosmoglobe homepage.

Markov Chain files[edit]

The full Markov Chains are provided on an HDF5 format, which have convenient IO wrappers for most commonly used programming languages (Python, C/C++, Fortran etc.). Three types of chain files are provided, corresponding to 1) full chains; 2) resampled CMB temperature chains; and 3) resampled CMB polarization chains. Each chain file stores each complete sample in a separate HDF dataset ("folder") marked "NNNNNN" (e.g., "000010" for sample number 10). The internal structure of each sample folder is summarized below for each chain file type. (Only main variables are listed below.)

Example traceplots from the BeyondPlanck chain file. For discussion of this plot, see Basyrov et al. (2022).
Full chains[edit]
Description Markov chains from main run that includes both astrophysical and instrumental parameters.
Filename BP_c000{1,2,3,4}_v2.h5
Number of samples per chain 1000
Size per chain file 1.7 TB
HDF path Description Specification Notes
ame/amp_alm (Unconvolved) a_lm's of AME component map (T-only) uK_RJ at 22 GHz Ordered according to libsharp convention
ame/nu_p_map AME nu_peak sky map GHz in flux density units See Andersen et al. (2022) for details
cmb/amp_alm (Unconvolved) a_lm's of CMB component map (T,Q,U) uK_cmb Ordered according to libsharp convention
cmb/sigma_l Angular power spectrum of CMB component map, sigma_l uK_cmb^2 Ordered as {TT,TE,TB,EE,EB,BB}
dust/amp_alm (Unconvolved) a_lm's of dust component map (T,Q,U) uK_RJ at 857 GHz for T; at 353 GHz for Q,U Ordered according to libsharp convention
dust/beta_map Thermal dust spectral index, beta_d, map (T,Q,U) Unitless Spatially constant but variable
dust/T_map Thermal dust temperature, T_d, map (T,Q,U) Kelvin Fixed to Planck PR4; does not vary with sample
ff/amp_alm (Unconvolved) a_lm's of free-free component map (T-only) uK_RJ at 40 GHz Ordered according to libsharp convention
md/{channel label} 4-element array with {monopole, d_x, d_y, d_z} corrections Same units as respective frequency channel Only monopoles and Haslam dipoles are non-zero
radio/amp Radio source amplitude per object (T-only) mJy at 30 GHz Ordered according to BP/Planck Commander point source catalog
radio/spec_ind Radio source spectral index per object (T-only) Unitless
synch/amp_alm (Unconvolved) a_lm's of synchrotron component map (T,Q,U) uK_RJ at 408 MHz in T; at 30 GHz in Q,U Ordered according to libsharp convention
synch/beta_map Synchrotron spectral index, beta_s, sky map Unitless See Svalheim et al. (2022) for details
tod/{channel label}/1Hz_ampl 1Hz amplitude per radiometer and PID Volt
tod/{channel label}/1Hz_temp Binned 1Hz template per radiometer Unitless Binned between 0 and 1 sec
tod/{channel label}/accept Accept flag per radiometer and PID Unitless 0 = rejected, 1 = accepted
tod/{channel label}/chisq Reduced normalized chisq per radiometer and scan sigma See Ihle et al. (2022) for details
tod/{channel label}/gain Total gain per radiometer and scan V/K See Gjerløw et al. (2022) for details
tod/{channel label}/map Frequency map for given channel (T,Q,U) uK_cmb See Basyrov et al. (2022) for details
tod/{channel label}/rms Frequency standard devivation map for given channel (T,Q,U) uK_cmb See Basyrov et al. (2022) for details
tod/{channel label}/xi_n Noise power spectral density parameters for radiometer and PID {sigma_0, alpha, fknee, A_s} See Ihle et al. (2022) for details
Resampled CMB temperature chains[edit]
Description Markov chains from CMB temperature analysis. These are produced by imposing a CMB confidence mask on the CMB component, while fixing instrumental and (most of the) astrophysical parameters at the values sampled in the main run. Each CMB temperature sample in these chain represents one in-painted Gaussian constrained realization with full-sky coverage. These samples are useful for CMB temperature analysis. See Colombo et al. (2022) and Paradiso et al. (2022) for details.
Filename BP_c000{1,2,3,4}_Tresamp_v2.h5
Number of samples per chain 1000
Size per chain file 15 GB
HDF path Description Specification Notes
cmb/amp_alm (Unconvolved) a_lm's of CMB component map (T,Q,U) uK_cmb Ordered according to libsharp convention
cmb/sigma_l Angular power spectrum of CMB component map, sigma_l uK_cmb^2 Ordered as {TT,TE,TB,EE,EB,BB}
cmb/D_l Ensemble averaged (theory) angular power spectrum of CMB component map, D_l uK_cmb^2 Ordered as {TT,TE,TB,EE,EB,BB}
Resampled CMB polarization chains[edit]
Description Markov chains from CMB polarization analysis. These are produced by resampling the CMB a_lms for l <= 64, while fixing instrumental and (most of the) astrophysical parameters at the values sampled in the main run. These samples are useful for low-resolution CMB polarization analysis. See Colombo et al. (2022) and Paradiso et al. (2022) for details.
Filename BP_c000{1,2,3,4}_Presamp_v2.h5
Number of samples per chain 50.000
Size per chain file 2.3 GB
HDF path Description Specification Notes
cmb_lowl/amp_alm (Unconvolved) a_lm's of CMB component map (T,Q,U), lmax = 64 uK_cmb Ordered according to libsharp convention

Frequency maps[edit]

BeyondPlanck frequency maps at 30, 44 and 70 GHz. For further discussion, see Basyrov et al. (2022).

The BeyondPlanck frequency FITS maps are produced by averaging individual frequency map samples over Gibbs iterations, and thus correspond to posterior mean maps. We note that error propagation with these maps is challenging, and these are primarily provided for visualization and comparison purposes. For precision scientific analysis, operating with the individual samples provided in the chain files is highly encouraged to propagate errors properly.

Note 1: Unlike Planck DR3, but similar to Planck PR4, the BeyondPlanck frequency maps retain the CMB Solar dipole. Note 2: Unlike Planck, but similar to WMAP, the BeyondPlanck frequency maps retain the relativistic kinematic quadrupole. Instead of subtracting this signal, it is included as an additional component in the signal model.

Sky map file data structure
Filename = BP_{FREQ}_IQU_n{NSIDE}_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_cmb The signal intensity map
Q_MEAN Real*4 uK_cmb The signal Stokes Q map
U_MEAN Real*4 uK_cmb The signal Stokes U map
I_RMS Real*4 uK_cmb The signal intensity white noise RMS
Q_RMS Real*4 uK_cmb The signal Stokes Q white noise RMS
U_RMS Real*4 uK_cmb The signal Stokes U white noise RMS
I_STDDEV Real*4 uK_cmb The signal intensity posterior std (~ systematic uncertainty)
Q_STDDEV Real*4 uK_cmb The signal Stokes Q posterior std (~ systematic uncertainty)
U_STDDEV Real*4 uK_cmb The signal Stokes U posterior std (~ systematic uncertainty)
Keyword Data Type Value Description
PIXTYPE string HEALPIX
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 512 or 1024 Healpix Nside
FIRSTPIX Int*4 0 First pixel number
LASTPIX Int*4 3145727 or 12582911 Last pixel number
FREQ string nnn The frequency channel

Component maps[edit]

BeyondPlanck CMB map. Rows show Stokes T, Q, and U, while columns show posterior mean and standard deviation. For further discussion, see Colombo et al. (2022).
Full-resolution raw posterior mean CMB map
Filename = BP_cmb_IQU_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_cmb The CMB intensity map
Q_MEAN Real*4 uK_cmb The CMB Stokes Q map
U_MEAN Real*4 uK_cmb The CMB Stokes U map
I_STDDEV Real*4 uK_cmb The CMB intensity posterior std
Q_STDDEV Real*4 uK_cmb The CMB Stokes Q posterior std
U_STDDEV Real*4 uK_cmb The CMB Stokes U posterior std
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
Low-resolution raw posterior mean CMB map
Filename = BP_CMB_QU_map_n8_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
TEMPERATURE Real*4 uK_cmb Low-resolution CMB intensity map
Q-POLARIZATION Real*4 uK_cmb Low-resolution CMB Stokes Q map
U-POLARIATION Real*4 uK_cmb Low-resolution CMB Stokes U map
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 8 Healpix Nside
Single in-painted high-resolution CMB temperature sample
Filename = BP_cmb_resamp_I_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_Stokes Real*4 uK_cmb The CMB intensity map
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
Posterior mean AME map
Filename = BP_ame_I_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_RJ The AME intensity map
I_NU_P_MEAN Real*4 GHz AME nu_peak frequency mean
I_STDDEV Real*4 uK_RJ AME intensity posterior std
I_NU_P_STDDEV Real*4 GHz AME nu_peak frequency rms
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
FWHM Real*4 120 Smoothing FWHM in arcmin
NU_REF_T Real*4 22.0 AME reference frequency in GHz
Full-resolution raw posterior mean thermal dust map
Filename = BP_dust_IQU_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_RJ Posterior mean intensity map
Q_MEAN Real*4 uK_RJ Posterior mean Stokes Q map
U_MEAN Real*4 uK_RJ Posterior mean Stokes U map
P_MEAN Real*4 uK_RJ Posterior mean polarization amplitude, P = sqrt(Q^2 + U^2)
I_BETA_MEAN Real*4 Unitless Posterior mean spectral index, beta_d, in intensity
QU_BETA_MEAN Real*4 Unitless Posterior mean spectral index, beta_d, in polarization
I_T_MEAN Real*4 Kelvin Posterior mean dust temperature, T_d, in intensity
QU_T_MEAN Real*4 Kelvin Posterior mean dust temperature, T_d, in polarization
I_STDDEV Real*4 uK_RJ Posterior rms intensity map
Q_STDDEV Real*4 uK_RJ Posterior rms Stokes Q map
U_STDDEV Real*4 uK_RJ Posterior rms Stokes U map
P_STDDEV Real*4 uK_RJ Posterior mean polarization amplitude, P = sqrt(Q^2 + U^2)
I_BETA_STDDEV Real*4 Unitless Posterior rms spectral index, beta_d, in intensity
QU_BETA_STDDEV Real*4 Unitless Posterior rms spectral index, beta_d, in polarization
I_T_STDDEV Real*4 Kelvin Posterior mean rms temperature, T_d, in intensity
QU_T_STDDEV Real*4 Kelvin Posterior mean rms temperature, T_d, in polarization
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
FWHM Real*4 10 Smoothing FWHM in arcmin
NU_REF_T Real*4 545 Reference frequency in temperature in GHz
NU_REF_P Real*4 353 Reference frequency in polarization in GHz
Full-resolution raw posterior mean free-free map
Filename = BP_freefree_I_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_RJ Posterior mean intensity map
I_TE_MEAN Real*4 Kelvin Posterior mean electron temperature, T_e. (Fixed in current analysis)
I_STDDEV Real*4 uK_RJ Posterior rms intensity map
I_TE_STDDEV Real*4 Kelvin Posterior rms electron temperature, T_e. (Zero i in current analysis)
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
FWHM Real*4 30 Smoothing FWHM in arcmin
NU_REF_T Real*4 40.0 Reference frequency in temperature in GHz
Full-resolution raw posterior mean synchrotron map
Filename = BP_synch_IQU_n1024_v2.fits
EXTNAME = 'FREQ-MAP' : Data columns
Column Name Data Type Units Description
I_MEAN Real*4 uK_RJ Posterior mean intensity map
Q_MEAN Real*4 uK_RJ Posterior mean Stokes Q map
U_MEAN Real*4 uK_RJ Posterior mean Stokes U map
P_MEAN Real*4 uK_RJ Posterior mean polarization amplitude, P = sqrt(Q^2 + U^2)
I_BETA_MEAN Real*4 Unitless Posterior mean spectral index, beta_d, in intensity
QU_BETA_MEAN Real*4 Unitless Posterior mean spectral index, beta_d, in polarization
I_STDDEV Real*4 uK_RJ Posterior rms intensity map
Q_STDDEV Real*4 uK_RJ Posterior rms Stokes Q map
U_STDDEV Real*4 uK_RJ Posterior rms Stokes U map
P_STDDEV Real*4 uK_RJ Posterior mean polarization amplitude, P = sqrt(Q^2 + U^2)
I_BETA_STDDEV Real*4 Unitless Posterior rms spectral index, beta_d, in intensity
QU_BETA_STDDEV Real*4 Unitless Posterior rms spectral index, beta_d, in polarization
Keyword Data Type Value Description
COORDSYS string GALACTIC Coordinate system
ORDERING string RING Healpix ordering
NSIDE Int 1024 Healpix Nside
FWHM Real*4 60 Smoothing FWHM in arcmin
NU_REF_T Real*4 0.408 Reference frequency in temperature in GHz
NU_REF_P Real*4 30 Reference frequency in polarization in GHz

Ancillary data[edit]

CMB confidence masks[edit]
CMB confidence masks for temperature (top) and polarization (bottom). For further discussion, see Colombo et al. (2022).

The BeyondPlanck processing involves two different CMB confidence masks, with high and low resolution, respectively:

  • BP_CMB_I_analysis_mask_n1024_v2.fits -- CMB T-only analysis mask at Nside=1024
  • BP_CMB_QU_map_n8_v2.fits -- CMB T+P analysis mask at Nside=8

Both maps are defined in Galactic coordinates with HEALPix ring ordering.

Revised LFI bandpass profiles[edit]
Comparison of Planck (orange) and BeyondPlanck (blue) bandpasses for all 30, 44 and 70 GHz radiometers. For further discussion, see Svalheim et al. (2022).

As discussed by Zonca et al. (2010) and Svalheim et al. (2022), the official Planck LFI bandpasses measured from ground were affected by measurement errors. These have been partially mitigated in the updated BeyondPlanck processing, and the improved bandpass profiles are provided in the form of ASCII tables. Each file is called BP_bandpass_LFI_{radiometer}_v2.dat, and contains an array with {nu, tau} on each line. The symbol '#' indicates comments.

Additional information[edit]

  • BeyondPlanck was an effort to generalize the Planck-developed component separation code called Commander to also support time-domain analysis. The BeyondPlanck software is thus an integral part of the latest Commander3 code, which is available from the Cosmoglobe GitHub repository
  • The Commander documentation describes both the installation procedure and the Commander parameter file
  • The BeyondPlanck papers are published in a Special Issue of Astronomy & Astrophysics called "BeyondPlanck: end-to-end Bayesian analysis of Planck LFI"

Planck Legacy Archive

European Space Agency

(Planck) Low Frequency Instrument

Cosmic Microwave background

Flexible Image Transfer Specification

Full-Width-at-Half-Maximum

(Hierarchical Equal Area isoLatitude Pixelation of a sphere, <ref name="Template:Gorski2005">HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on the Sphere, K. M. Górski, E. Hivon, A. J. Banday, B. D. Wandelt, F. K. Hansen, M. Reinecke, M. Bartelmann, ApJ, 622, 759-771, (2005).