Standard Product Generation Pipeline

The SWS Standard Product Generation  (SPG) pipeline in the Off-line Processing System (OLP) is designed to reduce all SWS observations from telemetry data to the final data product, i.e. flux- and wavelength-calibrated spectra. SWS SPG will be installed at the ISO Science Operation Center. Normally within two weeks after the observation, SWS data will be processed by the SPG and distributed to the proposer. After the mission, all data may be reprocessed to take advantage of the improved understanding of the instrument and its calibration that will evolve during the mission.

All data production processes can run concurrently for differing telemetry data and products. Fig. 7.1 shows a simplified data flow diagram for the pipeline.

TDF_FIRST_SCAN performs a first pass through all TDF data for one revolution (TDF = Telemetry Distribution Format is the format in which data arrive at SOC). The primary tasks are :

• Group the TDF and establish boundaries between various observations and their parts. The Compact Status History  (CSH) and Executed Observation History  (EOH) files which are to be used by later processing stages are created.
• Assign the Uniform Time Key (UTK), i.e. the time measure used by ISO
• Extract the transparent data (TDATA) that transport information e.g. on proposer name and AOT name to the data processing

Information about the exact pointing direction and orientation of the ISO satellite during the observations is obtained from the Spacecraft Control Center and fed into the pipeline via the Aperture Pointing History (APH).

DERIVE_ERD  converts the telemetry data to Edited Raw Data (ERD), which contains all scientifically significant information, but is reformatted to meet the constrains of the Product Archive. The ERD is formatted as FITS table. Each observer will receive an ERD per observation. Ref. 2 contains a description of the ERD product.

DERIVE_SPD  processes the read-outs for all detectors from the Edited Raw Data on a one second level. Results are saved in the Standard Processed Data (SPD) file in chronological order of aquisition. DERIVE_SPD executes three main tasks:

• Split up the data from the AOT into one readout (the time between detector resets) measurements. Each type of measurement (darks, object data etc.) is then flagged, slopes are reconstructed from the read-outs, a RC correction is carried out and the data is flagged for glitches caused by ionizing radiation and for the location of the reset pulses.
• Associate each measurement with its precise wavelength.
• Finally one single type of wavelength calibrated but intensity uncalibrated data is generated, namely the detector voltage slope with its associated wavelength.

All SPD data will be stored as FITS table and will be distributed to the observer. Ref. 2 gives a full description of the SPD.

AUTO_ANALYSIS . It is the intention to let SWS Auto Analysis (AA) provide the user with a series of wavelength and intensity calibrated spectra. Spectra will be generated according to the specifications of the observation as laid down in the AOT. The resulting spectra will be available as FITS tables. The various steps in Auto Analysis are:

• The data are corrected as well as possible for time line dependent effects (e.g. detector memory effects).
• The dark current is subtracted.
• Multiple scans of the same wavelength range (up-down or up-up), as forseen by most AOTs for redundancy, are used to correct for e.g. short term drifts in the responsivity.
• The measured data are converted to calibrated fluxes.
• Finally all the data relevant for a requested spectrum are stored in a table containing the final calibrated spectrum, giving an array of flux densities [Jy] and the associated wavelength. This table is sorted by increasing wavelength.

Auto Analysis Results (AAR) will be distributed to the observers and will be suitable for further analysis with software that is not specific to ISO.

Both DERIVE_SPD and AUTO_ANALYSIS make use of calibration files (CAL-G) either created within the pipeline from internal calibrations, or created by the Calibration Analysis system outside the pipeline. They may also access results handed over by the RTA/QLA system via the quick look history file (QLH).