Multi-Extension FITS files

In this section, we will see how to:

  • handle a multi-extension FITS files.
  • add or remove extensions
  • retrieve data or header
  • save a multi-extension file
  • save a single Image or Table

Keys: astropy.io.fits

Open a multi-extension FITS

Import required Astropy modules to handle fits files

In [2]:
from astropy.io import fits
In [ ]:
 

Python modules

A module is a file containing Python definitions and statements.

The file name is the module name except for the suffix .py appended. Within a module, the module’s name is available as the value of the global variable __name__

A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement.

Modules can import other modules

A module can be aliased to a different name using the import xx as yy import statement

In [ ]:
import numpy as np

The Module Search Path

When a module named spam is imported, the interpreter:

  • first searches for a built-in module with that name.
  • if that is not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

    • the directory containing the input script (or the current directory when no file is specified).
    • the shell variable environment PYTHONPATH (a list of directory names, specified with the same syntax as the shell variable PATH).
    • The installation-dependent default.

Open multi-extension FITS

In [ ]:
from astropy.io import fits

Open the file: the .open() function doesn't load data and headers, it just gives us access to the file and allows us to view the file structure.

In [3]:
filename='data/mef_example.fits'
hdulist=fits.open(filename)
In [ ]:
 

Explore the hdulist object

When a FITS file is opened, an HDUList object is returned.

This is the top-level FITS object: a list-like collection of HDU objects.

An HDU (Header Data Unit) is the highest level component of the FITS file structure, consisting of a header and (typically) a data array or table.

In [4]:
hdulist
Out[4]:
[<astropy.io.fits.hdu.image.PrimaryHDU object at 0x7f5c9003bf60>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7f5c6b4f6e80>, <astropy.io.fits.hdu.table.BinTableHDU object at 0x7f5c6b505860>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7f5c6aa1f940>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7f5c6aa1f898>]
In [ ]:
 
In [29]:
%pprint
dir(hdulist)
Pretty printing has been turned OFF
Out[29]:
['__add__', '__class__', '__contains__', '__copy__', '__deepcopy__', '__delattr__', '__delitem__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_data', '_file', '_flush_resize', '_flush_update', '_in_read_next_hdu', '_open_kwargs', '_positive_index_of', '_read_all', '_read_next_hdu', '_readfrom', '_resize', '_save_backup', '_truncate', '_try_while_unread_hdus', '_verify', '_wasresized', 'append', 'clear', 'close', 'copy', 'count', 'extend', 'fileinfo', 'filename', 'flush', 'fromfile', 'fromstring', 'index', 'index_of', 'info', 'insert', 'pop', 'readall', 'remove', 'reverse', 'run_option', 'sort', 'update_extend', 'verify', 'writeto']
In [ ]:
 

IPython info of objects

In [6]:
hdulist?
In [ ]:
 

Python docstring

  • Python documentation strings (or docstrings) provide a convenient way of associating documentation with Python modules, functions, classes, and methods.

  • Unlike conventional source code comments the docstring should describe what the function does, not how.

In [7]:
hdulist.info?
In [ ]:
 

The FITS file structure

In [8]:
hdulist.info()
Filename: data/mef_example.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      12   (2155,)   float32   
  1  EXR2D         1 ImageHDU        17   (2155, 92)   float32   
  2  OBJ           1 BinTableHDU    180   103R x 9C   [1J, 1J, 1J, 1J, 1J, 1E, 1E, 1D, 1D]   
  3                1 ImageHDU         7   (37, 15)   uint8   
  4  EXR2D         1 ImageHDU         8   (37, 15)   uint8   
In [ ]:
 

Browse the extensions

by index (python indexes start from 0)

In [9]:
hdulist[1]
Out[9]:
<astropy.io.fits.hdu.image.ImageHDU at 0x7f5c6b4f6e80>

or by extention name

In [10]:
hdulist['EXR2D']
Out[10]:
<astropy.io.fits.hdu.image.ImageHDU at 0x7f5c6b4f6e80>

1 and EXR2D are the same extension

In [11]:
hdulist[1] is hdulist['EXR2D']
Out[11]:
True
In [ ]:
 

Remove extensions

Again: by index

In [12]:
exr2d = hdulist.pop(1)
In [ ]:
 

or by hdu name

In [13]:
obj = hdulist.pop('OBJ')
In [ ]:
 

Add extensions

Append extension at the end of the file

In [14]:
hdulist.append(exr2d)
In [15]:
hdulist.info()
Filename: data/mef_example.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      12   (2155,)   float32   
  1                1 ImageHDU         7   (37, 15)   uint8   
  2  EXR2D         1 ImageHDU         8   (37, 15)   uint8   
  3  EXR2D         1 ImageHDU        17   (2155, 92)   float32   

Or insert extension in a specific position

In [16]:
hdulist.insert(1, obj)
In [ ]:
 

Get the header

The HDU object has the .header attribute to handle FITS header, both for tables

In [17]:
obj.header
Out[17]:
XTENSION= 'BINTABLE'           / binary table extension                         
BITPIX  =                    8 / 8-bit bytes                                    
NAXIS   =                    2 / 2-dimensional binary table                     
NAXIS1  =                   44                                                  
NAXIS2  =                  103                                                  
PCOUNT  =                    0 / size of special data area                      
GCOUNT  =                    1 / one data group (required keyword)              
TFIELDS =                    9                                                  
TTYPE1  = 'SLIT    '           / label for field   1                            
TFORM1  = '1J      '           / data format of field: 4-byte INTEGER           
TTYPE2  = 'IFUSLIT_NO'         / label for field   2                            
TFORM2  = '1J      '           / data format of field: 4-byte INTEGER           
TTYPE3  = 'IFUFIBER_NO'        / label for field   3                            
TFORM3  = '1J      '           / data format of field: 4-byte INTEGER           
TTYPE4  = 'Y       '           / label for field   4                            
TFORM4  = '1J      '           / data format of field: 4-byte INTEGER           
TTYPE5  = 'OBJ_NO  '           / label for field   5                            
TFORM5  = '1J      '           / data format of field: 4-byte INTEGER           
TTYPE6  = 'OBJ_X   '           / label for field   6                            
TFORM6  = '1E      '           / data format of field: 4-byte REAL              
TTYPE7  = 'OBJ_Y   '           / label for field   7                            
TFORM7  = '1E      '           / data format of field: 4-byte REAL              
TTYPE8  = 'OBJ_RA  '           / label for field   8                            
TFORM8  = '1D      '           / data format of field: 8-byte DOUBLE            
TTYPE9  = 'OBJ_DEC '           / label for field   9                            
TFORM9  = '1D      '           / data format of field: 8-byte DOUBLE            
EXTNAME = 'OBJ     '                                                            
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                

and for images

In [18]:
hdulist[0].header
Out[18]:
SIMPLE  =                    T / file does conform to FITS standard             
BITPIX  =                  -32 / number of bits per data pixel                  
NAXIS   =                    1 / number of data axes                            
NAXIS1  =                 2155 / length of data axis 1                          
EXTEND  =                    T / FITS dataset may contain extensions            
COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H 
CRVAL1  =                4500.                                                  
CRPIX1  =                   1.                                                  
CTYPE1  = 'WAVE    '                                                            
CDELT1  =     2.55299997329712                                                  
CUNIT1  = 'Angstrom'                                                            
In [ ]:
 

Get data

The .data attribute gives us access to tables data

In [19]:
obj.data
Out[19]:
FITS_rec([( 1, 0, 0,   0, 1, -119.608116 ,  7.42040e+01, 0., 0.),
          ( 2, 0, 0,   1, 1, -115.60632  , -1.09026e+02, 0., 0.),
          ( 3, 0, 0,   2, 1, -115.35641  ,  1.42585e+02, 0., 0.),
          ( 4, 0, 0,   3, 1, -113.2135   , -1.01476e+02, 0., 0.),
          ( 4, 0, 0,   4, 2, -111.777954 , -1.01476e+02, 0., 0.),
          ( 4, 0, 0,   5, 3, -108.6676   , -1.01476e+02, 0., 0.),
          ( 5, 0, 0,   6, 1, -108.60712  , -5.87600e+00, 0., 0.),
          ( 6, 0, 0,   7, 1, -110.067444 ,  8.50790e+01, 0., 0.),
          ( 7, 0, 0,   8, 1, -103.89667  , -7.02480e+01, 0., 0.),
          ( 8, 0, 0,   9, 1, -104.46953  ,  2.90410e+01, 0., 0.),
          ( 9, 0, 0,  10, 1,  -98.06541  , -3.51880e+01, 0., 0.),
          (10, 0, 0,  11, 1,  -99.7153   ,  4.45260e+01, 0., 0.),
          (11, 0, 0,  12, 1,  -92.164055 , -1.10603e+02, 0., 0.),
          (12, 0, 0,  13, 1,  -92.60097  , -5.74900e+00, 0., 0.),
          (13, 0, 0,  14, 1,  -92.32721  ,  1.14094e+02, 0., 0.),
          (14, 0, 0,  15, 1,  -87.38498  , -1.05911e+02, 0., 0.),
          (14, 0, 0,  16, 2,  -86.18945  , -1.05911e+02, 0., 0.),
          (15, 0, 0,  17, 1,  -85.2454   ,  3.15660e+01, 0., 0.),
          (16, 0, 0,  18, 1,  -86.30322  ,  1.42199e+02, 0., 0.),
          (17, 0, 0,  19, 1,  -79.98973  , -8.52750e+01, 0., 0.),
          (18, 0, 0,  20, 1,  -80.94561  ,  3.08900e+01, 0., 0.),
          (19, 0, 0,  21, 1,  -75.25018  , -4.01500e+01, 0., 0.),
          (20, 0, 0,  22, 1,  -75.983315 ,  6.41290e+01, 0., 0.),
          (21, 0, 0,  23, 1,  -75.10154  ,  1.54171e+02, 0., 0.),
          (22, 0, 0,  24, 1,  -68.35159  , -1.59720e+01, 0., 0.),
          (23, 0, 0,  25, 1,  -64.41236  ,  6.87920e+01, 0., 0.),
          (24, 0, 0,  26, 1,  -59.68167  , -9.32580e+01, 0., 0.),
          (25, 0, 0,  27, 1,  -60.748634 ,  1.22820e+01, 0., 0.),
          (25, 0, 0,  28, 2,  -58.83927  ,  1.22820e+01, 0., 0.),
          (26, 0, 0,  29, 1,  -62.617672 ,  1.28410e+02, 0., 0.),
          (26, 0, 0,  30, 2,  -60.349724 ,  1.28410e+02, 0., 0.),
          (27, 0, 0,  31, 1,  -45.9311   , -4.24500e+00, 0., 0.),
          (28, 0, 0,  32, 1,  -46.021385 ,  1.26591e+02, 0., 0.),
          (29, 0, 0,  33, 1,  -41.52     , -1.10012e+02, 0., 0.),
          (30, 0, 0,  34, 1,  -41.414776 ,  8.69400e+00, 0., 0.),
          (31, 0, 0,  35, 1,  -42.036407 ,  1.02057e+02, 0., 0.),
          (32, 0, 0,  36, 1,  -38.561604 , -8.61700e+01, 0., 0.),
          (32, 0, 0,  37, 2,  -35.577747 , -8.61700e+01, 0., 0.),
          (33, 0, 0,  38, 1,  -30.18095  , -1.11533e+02, 0., 0.),
          (34, 0, 0,  39, 1,  -32.984318 ,  1.11016e+02, 0., 0.),
          (34, 0, 0,  40, 2,  -30.71782  ,  1.11016e+02, 0., 0.),
          (35, 0, 0,  41, 1,  -26.02951  ,  9.48200e+00, 0., 0.),
          (36, 0, 0,  42, 1,  -21.624432 , -8.06200e+01, 0., 0.),
          (37, 0, 0,  43, 1,  -21.524958 ,  2.99130e+01, 0., 0.),
          (38, 0, 0,  44, 1,  -14.595522 , -7.33290e+01, 0., 0.),
          (39, 0, 0,  45, 1,  -16.021965 ,  1.64620e+01, 0., 0.),
          (40, 0, 0,  46, 1,   -8.362046 , -9.92610e+01, 0., 0.),
          (41, 0, 0,  47, 1,   -9.293859 , -2.10640e+01, 0., 0.),
          (42, 0, 0,  48, 1,   -9.04891  ,  6.03460e+01, 0., 0.),
          (43, 0, 0,  49, 1,   -2.5964928, -3.59340e+01, 0., 0.),
          (44, 0, 0,  50, 1,   -4.033945 ,  1.32130e+02, 0., 0.),
          (45, 0, 0,  51, 1,   -0.4278739, -5.33250e+01, 0., 0.),
          (45, 0, 0,  52, 2,    1.3605044, -5.33250e+01, 0., 0.),
          (46, 0, 0,  53, 1,    1.3615531,  3.42010e+01, 0., 0.),
          (46, 0, 0,  54, 2,    3.38824  ,  3.42010e+01, 0., 0.),
          (47, 0, 0,  55, 1,    9.226427 , -5.03980e+01, 0., 0.),
          (48, 0, 0,  56, 1,    8.116842 ,  1.42469e+02, 0., 0.),
          (49, 0, 0,  57, 1,   14.850358 , -6.54050e+01, 0., 0.),
          (50, 0, 0,  58, 1,   13.669239 ,  9.75700e+01, 0., 0.),
          (51, 0, 0,  59, 1,   18.910612 ,  1.00303e+02, 0., 0.),
          (52, 0, 0,  60, 1,   23.899364 ,  2.87140e+01, 0., 0.),
          (53, 0, 0,  61, 1,   23.2754   ,  1.34514e+02, 0., 0.),
          (54, 0, 0,  62, 1,   27.753693 , -8.40640e+01, 0., 0.),
          (55, 0, 0,  63, 1,   27.485924 ,  2.12570e+01, 0., 0.),
          (56, 0, 0,  64, 1,   25.082659 ,  1.22246e+02, 0., 0.),
          (56, 0, 0,  65, 2,   28.53829  ,  1.22246e+02, 0., 0.),
          (57, 0, 0,  66, 1,   33.019928 , -1.44330e+01, 0., 0.),
          (58, 0, 0,  67, 1,   33.97993  ,  6.49340e+01, 0., 0.),
          (59, 0, 0,  68, 1,   33.395218 ,  1.48212e+02, 0., 0.),
          (60, 0, 0,  69, 1,   45.593807 , -5.39890e+01, 0., 0.),
          (60, 0, 0,  70, 2,  -53.989    ,  1.40130e-45, 0., 0.),
          (61, 0, 0,  71, 1,   45.336937 ,  4.07030e+01, 0., 0.),
          (62, 0, 0,  72, 1,   46.297855 ,  1.28767e+02, 0., 0.),
          (63, 0, 0,  73, 1,   50.872593 , -7.53470e+01, 0., 0.),
          (64, 0, 0,  74, 1,   51.167118 ,  4.74570e+01, 0., 0.),
          (65, 0, 0,  75, 1,   56.663136 ,  3.70050e+01, 0., 0.),
          (66, 0, 0,  76, 1,   54.76473  ,  1.31123e+02, 0., 0.),
          (67, 0, 0,  77, 1,   58.6325   , -7.87050e+01, 0., 0.),
          (67, 0, 0,  78, 2,   60.77545  , -7.87050e+01, 0., 0.),
          (68, 0, 0,  79, 1,   60.430305 ,  7.51200e+01, 0., 0.),
          (68, 0, 0,  80, 2,   61.85958  ,  7.51200e+01, 0., 0.),
          (69, 0, 0,  81, 1,   63.805725 , -3.54450e+01, 0., 0.),
          (69, 0, 0,  82, 2,   66.42547  , -3.54450e+01, 0., 0.),
          (70, 0, 0,  83, 1,   66.04384  ,  6.51580e+01, 0., 0.),
          (71, 0, 0,  84, 1,   65.72256  ,  1.50485e+02, 0., 0.),
          (72, 0, 0,  85, 1,   72.873436 , -1.12204e+02, 0., 0.),
          (73, 0, 0,  86, 1,   74.22504  ,  6.15300e+00, 0., 0.),
          (74, 0, 0,  87, 1,   71.02607  ,  8.54410e+01, 0., 0.),
          (75, 0, 0,  88, 1,   80.73261  , -3.86400e+01, 0., 0.),
          (76, 0, 0,  89, 1,   80.90338  ,  1.02864e+02, 0., 0.),
          (77, 0, 0,  90, 1,   83.022705 , -1.09812e+02, 0., 0.),
          (77, 0, 0,  91, 2,   84.68768  , -1.09812e+02, 0., 0.),
          (78, 0, 0,  92, 1,   88.6741   , -1.25280e+01, 0., 0.),
          (79, 0, 0,  93, 1,   86.65988  ,  6.82050e+01, 0., 0.),
          (80, 0, 0,  94, 1,   93.39009  , -4.89810e+01, 0., 0.),
          (81, 0, 0,  95, 1,   91.45818  ,  4.55620e+01, 0., 0.),
          (81, 0, 0,  96, 2,   93.00592  ,  4.55620e+01, 0., 0.),
          (82, 0, 0,  97, 1,   99.217575 ,  3.23130e+01, 0., 0.),
          (83, 0, 0,  98, 1,   97.5867   ,  1.35900e+02, 0., 0.),
          (84, 0, 0,  99, 1,  103.36665  , -8.61290e+01, 0., 0.),
          (85, 0, 0, 100, 1,  104.00345  ,  2.04830e+01, 0., 0.),
          (86, 0, 0, 101, 1,  108.91283  , -6.05080e+01, 0., 0.),
          (87, 0, 0, 102, 1,  108.75126  ,  3.57010e+01, 0., 0.)],
         dtype=(numpy.record, [('SLIT', '>i4'), ('IFUSLIT_NO', '>i4'), ('IFUFIBER_NO', '>i4'), ('Y', '>i4'), ('OBJ_NO', '>i4'), ('OBJ_X', '>f4'), ('OBJ_Y', '>f4'), ('OBJ_RA', '>f8'), ('OBJ_DEC', '>f8')]))

or images data

In [20]:
exr2d.data
Out[20]:
array([[2., 2., 2., ..., 2., 2., 2.],
       [2., 2., 2., ..., 2., 2., 2.],
       [2., 2., 2., ..., 2., 2., 2.],
       ...,
       [2., 2., 2., ..., 2., 2., 2.],
       [2., 2., 2., ..., 2., 2., 2.],
       [2., 2., 2., ..., 2., 2., 2.]], dtype=float32)
In [ ]:
 

Save the file

We can save the whole HDU list

In [21]:
outfile='tmp/hdu_list.fits'
hdulist.writeto(outfile, overwrite = True)
In [22]:
hdulist.writeto?

Or save just a single HDU, like a table

In [23]:
outfile='tmp/hdu_table.fits'
hdulist['OBJ'].writeto(outfile, overwrite = True)
In [ ]:
 

or an image

In [24]:
outfile='tmp/hdu_image.fits'
hdulist['EXR2D'].writeto(outfile, overwrite = True)

NOTE: In this case, since we are writing an ImageHDU (and not a PrimaryHDU), the saved file contains an empty PRIMARY and the image is saved as the second HDU

In [25]:
exr2d.writeto('tmp/hdu_image.fits', overwrite = True)

If we want to save the image as the Primary extension, we must create a PrimaryHDU from scratch using the data and header from the image we want to save

In [26]:
primary = fits.PrimaryHDU(data=hdulist['EXR2D'].data, 
                          header=hdulist['EXR2D'].header)
primary.writeto('tmp/primary_image.fits', overwrite = True)
In [ ]:
 

Close the file

After we are done with the previously opened file, we have to close it with the .close() method

In [27]:
hdulist.close()
In [ ]:
 

With statement

We can avoid closing the file manually by using the with statement to open the file. The with syntax is showed in the next code block.

In [28]:
with fits.open(filename) as hdulist:
    exrd2d=hdulist['EXR2D'].copy()
In [ ]:
#That is equivalent to
hdulist=fits.open(filename)
exrd2d=hdulist['EXR2D'].copy()
hdulist.close()

After exiting the with scope, the file will be closed automatically. This is the preferred way to open a file in Python, because it will close the file even if an exception happens.

In [ ]: