Frequently used software (data reduction III)

First a distinction to do which operating system you use or want to use, Windows or Linux.

Linux can be simulated on Windows platforms with the software „cygwin„. For cygwin exist a specially compiled version of MIDAS. However, scripts running slower under cygwin than in an original Linux operating system (Ubuntu, openSuse ,…). Under Vista I had recently trouble even start the cygwin X-system in.

Additional opportunities to emulate the operating system such as VMware, I will not go into because lack of own experience here.

MIDAS (in Linux)

The European astronomers use ESO MIDAS, that can run in Linux (and some other Unix related platforms). A Windows version doesn’t exist. It is freely downloadable from the ESO-website.

MIDAS consists of a multitude of scripts and environments (called „context„). Starting with the examination of the CCD, imaging, complex evaluations such as time series analysis, but also specific evaluation scripts for spectroscopy of slit spektrographs and Echelle spectrographs – everything is to be found in MIDAS. However, the scripts are controlled in a Linux console by commands. It is for the discerning beginner difficult to get familiar with Midas because of the unusual syntax. Therefore our friend Guenter Gebhard wrote scripts, which can run the complex data reduction process automatically in the background, so the number of commands entered by the user is clearly reduced. Much is done via mouse click. These marvels of data reduction software, you can download freely here under „Data Reduction“, also english versions. The context OPA is designed for slitless spectrographs, and SMS is for slit spektrographs because there are fundamental differences in the evaluation routine between the two types of spectrographs. I know of no program that so easily and quickly transfers frame series into the calibrated spectrum: With some commands in SMS it is done within a few minutes. There are still optional commands for determining the resolution, normalization variants, measurements of the equivalent widths of lines, storage of spectra (diagrams) in various formats. png,. bmp,. pdf … and much more – depending on what you want.

SMS and OPA under MIDAS log every step and save the intermediate results (spectra, medians of darks, etc.), so that an evaluation can be carried out and re-examination in minute details, if one so wishes. This does not provide non-professional software from the amateur scene in this strict form. Here we must believe the results and if anything unusual occurs, nobody knows what happened where.

If you have never worked with Linux, there are some hurdles to overcome. First, the Linux operating system is to install, what can be done in parallel with Windows on a PC (in different partitions). We then have both operating systems on the same PC. See the Linux distributions (openSuse, Ubuntu etc.) to give the installation opportunity. If Windows occupies the entire hard drive (it just), one must first reduce the size of the Windows partition. Under Vista it is possible that in the Control Panel / Administrative Tools / Disk Management using the command „Shrink Volume to do.“ For a modern Linux system, you should have at least 10 GB free space. If you want to store many images, perhaps even more. It depends on the available disk capacity of your PC or laptop computers. Ubuntu has the advantage, that you are able to store and read files in the Windows partition. The Linux operating system itself requires more than 2 GB even if graphical interface (KDE or GNOME) is similar to the one you want in Windows.

If Linux is installed the next step can be the installation of MIDAS. There are instructions for installation in the ESO website. The latest version is the preferable one.

And the third step can be the installation of the OPA and / or SMS scripts.

A very useful introduction to MIDAS gives Günter Gebhard at his website: „First Steps in MIDAS„. A good supplement can be found here (some good tips on everyday with MIDAS). For detailed instructions see the documentation of Midas.


SMS (ESO-MIDAS in Linux for the evaluation of frame series taken by a long slit spektrograph)

Assuming the mechanical stability of the spectrograph is sufficient that the beam is not changed within the spectrograph during a series of recordings, then are the spectral bands of individual images at the same place, both in the direction of the dispersion (x axis) and transverse to (y axis). Under these circumstances, we may add directly the extracted spectra in the data reduction routine. On the other hand is inserted in case of slitless spectrograph, a treatment step in which the extracted 1D-spectra in the x-axis are shifted so that they all fit perfectly superimposed (correlation).

SMS does the data reduction, without having to make flats, bias or dark frames available. Such should indeed be taken into account, but it also goes without. Bias and dark current are included in every single shot in the background next to the spectra strip. The script generates for each pixel column of each record a polynomial for the background (which is composed of readout noise from sky background, stray light, dark current and bias) and subtract this „background“ model on every shot. Cosmics and hot pixels are statistically detected when individual pixels differ materially from the flanking pixels. These are automatically attenuated by filters. The ability to routinely handle all this in one step without the need for manual intervention I know of no other program in this consequence. Therefore I use generally SMS for routine reduction of my series recordings taken with the slit spektrograph LHIRES III .

After the first three commands (which include the use of a reference spectrum with a calibration lamp) the calibrated sum spectrum is produced and most of the work is actually done. Then we can normalize the spectrum to a point, in order to move the values of the relative fluxes (y-axis of the graph of the spectrum). With this „1 point normalization“ is only the y-scale scaled (mathematically is it a linear transformation, therefore the measured spectrum is not changed in itself), different spectra will be better comparable.

The normalization of the spectrum with the continuum of the star is also a usual technique. Here, the user must define the nodal points that define the continuum of the stellar spectrum by mouse clicks. If wide absorption lines in the spectrum are present, or „broad“ blends of many lines, then this definition of points is subjectively „personally flavoured“. Normalized spectra are therefore not a pure measurement product, but they are influenced by the user. So I save my test results using always the original sum spectrum or the one-point normalized spectrum. So I can fall back years later spectra in the original version at other scientific issues and evaluate arbitrary. From a continuum normalized spectrum the original spectrum can not be reconstructed, unless you have stored the corresponding continuum function (a spline interpolation points of the manually entered).

It has yet to determine the possibility of the S / N ratio of the spectrum (signal to noise ratio), to measure equivalent widths, labelling the graph, save the graphs (. gif, or. jpg or other formats) and much more.

OPA ( ESO-MIDAS in Linux for the evaluation of frame series taken by a slitless spectrograph)

The Midas sxript OPA is designed for the evaluation of frame series made by using a slitless spectrograph. Similar to SMS, with a few commands the recordings and the accompanying darks, and flats, and biasis are operated and a corrected sum spectrum results. In OPA should be at least darks used, recorded under the same conditions and with the same exposure time as the object images. Flats should be included if they are of good quality. Entering a calibration light frame is not possible. Instead, the extracted 1D spectra can be conveniently superimposed by means of individual correlations and added to a sum spectrum, which is then calibrated. The calibration is done by identifying 2 ore more lines of the spectrum or at least 1 line and using the known dispersion. The calibration is then carried out, however, in the reference system of the object. For a absolute calibration, e.g for measuring radial velocities, must be calibrated with lines that were measured in the rest frame of the observer. Fortunately, there are such in spectra: The terrestric lines that are formed in the atmosphere (bands of oxygen and water lines). That succeeds particularly well for the H alpha line of neutral hydrogen at 656.3 nm after the calibration can be re-normalized. There are a number of scripts available to variables such as the S / N, equivalent widths of lines, V / R ratio (in Be stars) and labeled graphs production.

To see the physically very sharp water lines we need resolution of R = 6,000 in minimum, better with R > 10,000. The terrestric absorption lines have physical widths in the range of a few hundredths of angstrom. Their line profile in the measured spectrum therefore reflects the mapping function of the spectrograph. They can be applied directly to deduce the resolution obtained.


VSpec (Windows)

Vspec is written by Valery Desnoux and freely available for Windows platforms, with which a (previously produced by IRIS) 2D-sum spectrum, or also an untreated raw frame can be extracted and the spectrum is shown as graph. The whole software is working on a graphical basis, almost all commands are done by mouse clicks. Therefore, the software is very well suited for the beginner. To submit the spectra a scalable window is arranged around the spectrum stripe and click – is already extracted the 1D spectrum. It can also include reference spectra (such as the Neon spectra of a slit spektrograph). In order to calibrate the 1D spectrum you can then take many tools from the toolbox:

  1. Reference spectra of a typical star of the same spectral class, taken from an extensive spectrum library
  2. Theoretical lines from one line catalog which contains all the essential elements and ions. A very handy tool (!), especially for the beginner who has trouble to recognize the line pattern of a spectrum and identify it.

Once calibrated, a spectrum can be easily normalized by mouse clicks on the continuum, if you want it also automatically. The determination of equivalent widths is a breeze, and other mathematical operations with the spectrum, such as the correction with a response function of the measuring device, obtained in a spectroscopic standard star. Even the direct online access to data from the SIMBAD database and others are possible.

All in all, a successful software that has been substantially improved over the last 5 years, in particular also with regard to bugs. I can only recommend it to any beginner, so that the entry into the data reduction is to create quickly. One drawback should not be denied: The programm is a cookbook for the amateur scene and there are not the controlling possibilities as in Midas.
IRIS (Windows)

IRIS, written by Christian Buil, is an free image editing routine with a focus on the reduction of spectra recording series. Cosmics, removing the dark current, the sky background correction, etc. At the end, a corrected 2D sum spectrum is available, which is then extracted and processed by VSpec. It can, however, also perform the extraction already in IRIS.



Links to other software are found on the website of the Spectroscopy Section of the VDS server. But I think the really important, I have listed here. I use both software complexes: From time to time IRIS / VSpec for a quick overview, or for line identifications. For the daily routine evaluations I use SMS and sometimes more scientifically oriented evaluations in MIDAS in the form of individual scripts. At first glance looks like that IRIS and VSpec offer everything imaginable, however the professional MIDAS has the most mighty power.