Derive-SPDs main input is Edited Raw Data (ERD), 24 Hz data containing detector readout voltages of the target's flux, of internal flux standards, of wavelength calibration exposures, of dark currents and of housekeeping data. Derive-SPD also reads in pointing and instrument status files.
For each detector, all data for one full reset interval are combined to a single current estimate at single wavelength and are then saved in the Standard Processed Data (SPD) file for that AOT. No attempt is made within Derive-SPD to average measurements of the same type (e.g. dark current measurements) or to do processing on any time scale longer than one reset interval.
A basic overview of the Derive-SPD processing is given in figure 8.1 and the ERD and SPD files are described in sections 9.2.1 and 9.3.1.
Each measurement for one reset interval of a given AOT is saved separately inside the same SPD product in chronological order of acquisition. The motivation for this is to offer to the observer the possibility of judging the stability and repeatability of SWS measurements. There is an one-to-one correspondence between AOT's and SPD files. That is, one AOT produces one SPD file.
The individual processing steps are described as follows:
Data is read in for one reset interval (SW and/or LW reset interval). As discussed above, each AOT generates different types of data: astronomical measurements from the source, measurements from the spectral reference sources (grating and FP), measurements with the calibrators and dark currents. Each of these different types of data has a different physical meaning and serves a different purpose. Hence, the data must be flagged according to their type.
Each ERD data record is tagged with a Instrument Time Key (ITK) which points toward a unique record in the SSTA Compact Status History (CSH) file (see section 9.2.2). Cross-correlating the ERD ITK with the CSH, Executed Observation History per ICS (EOHI) and General Housekeeping (GEHK) files (both described in the ISO Satellite & Data Manual) allows an unambiguous determination of the status of the SWS instrument at the time of the measurement and therefore of the type of the data recorded and the actual wavelength range covered (to be verified during the wavelength check stages).
This marks data affected by capacitor being short circuited at the end of the reset interval or data out of limits. To do this it uses Cal-G files 3 and 4.
The detector readouts voltages of between -10 and +10 V are converted into a bit number ranging from 0 to 4095. The midbit is the bit value that corresponds to 0 Volts, approximately 2048, and is different for each detector. This data is stored in Cal-G file 2A.
This module was only active for OLP Version 4.3. During testing, unusual results were obtained that have not been explained yet. Therefore from OLP V5.0 onwards Cal-G file 2A has been filled with neutral values exactly half of 4095, i.e. 2047.50.
Cal-G 2B contains information describing the shape of the pulse at the end of the reset interval. This calfile is used to subtract the effect of the reset pulse from the data. Cal-G file 5 is also used in this.
This module was only active for OLP Version 4.3. During testing, unusual results were obtained that have not been explained yet. Therefore from OLP V5.0 onwards Cal-G file 2B has been filled with zeros that result in this module doing nothing.
The amplifier chain contains a high pass filter. For a constant voltage gradient input the response of the amplifier is to asymptotically approach a constant value, i.e.
 |
(8.1) |
where V(t) is the output voltage at time t, Vo the input voltage applied on the capacitance due to the accumulation of charges on the detector, and to the detector time constant. In order to later reconstruct the true photo-current, it is first necessary to linearize the ramps, i.e. determine the actual input voltages Vo(t) applied on the detector.
Knowing the RC time constant to, it is straightforward to invert the above formula and derive the input voltage. Each detector i has its own time constant to(i), stored in Cal-G 2.
Due to parasitic capacitance between neighbouring detectors any signal in one grating detector will `leak' to other detectors (primarily the adjacent ones) in the same array. This `leak' to the adjacent detector is of the order of 10%.
Assuming the detector response is a linear function of the intensity of the signal, a set of cross-talk matrices can be determined with constant correction factors. The read-outs D of each individual detector j in the detector array may then be corrected by applying the following formula:
Assuming an error matrix Ei,j on the individual values of Fi,j can be derived at the same time as the cross-talk matrix, it is then straightforward to propagate these uncertainties.
These errors are not calculated in the generation of the array Fi,j and hence their effects on D are not calculated. Also, if the dependance of Fi,j on signal intensity is not linear equation 8.2 will not correctly describe the necessary correction and so any errors will be larger than given by equation 8.3.
Glitches (ref section 5.7) are recognized in the
differentiated voltages. The median of the 24 Hz bit values is computed and
used, in conjunction with a value 
held in Cal-G 6, to
define how high a glitch might be. If any differentiated 24 Hz bit values are
higher than this value they, and N-samples (currently one, also defined in
Cal-G file 6) before and after the glitch, are marked as being affected by a
glitch. If the glitch is so strong that the detector gets saturated all
measurements up to the end of the ramp are discarded.
If the glitch is not too strong the following measurements do not have to be discarded. In the case of one or two glitches in a slope the software tries to average the partial slopes between the glitches to one slope. One has to be aware, however, that the slope after a glitch is generally higher than the slope before a glitch (especially for a large glitch).
It is not possible to compute reasonable slopes for reset intervals affected by more than three glitches. The current pipeline does not flag this data as invalid, although the calculation of the slopes is imperfect.
The accuracy of deglitching depends not only on how well a glitch can be identified, but also on the effects of the glitch with time (the so called glitch-tails). Studies of slope variations after glitches are ongoing
The SWS Glitch History (SWGH) file contains a list of all glitches detected during processing along with a short characterisation, such as glitch height, skipped samples, etc. For a description of this file see section 9.3.2
This computes the grating and FP positions.
A least-square fit is applied to the data taken over one reset interval in order to estimate its slope S.
| V | = | S x t + O | (8.4) |
| S | = | I x k | (8.5) |
k is a constant which depends on the internal capacitance, put explicitly to be a constant 1. This implies that I has units of Amp per Farad. It does not need to be explicitly measured as it cancels out when the relative flux calibration is applied.
O is the charge offset at the zero-point of each ramp.
The time t is counted in seconds since the start of the ramp.
This module uses the glitch information generated by the deglitching algorithm above to raise a flag when the slopes are affected by 1, 2 or more glitches, are saturated or when there is no data (see SWSPFLAG in Table 9.8 for details). Because of this the routine uses Cal-G file 6.
Refer to section 7.3 for more details on flux calibration.
The electronics amplify the detector read-out voltages with a commandable gain factor before the analog to digital (AD) conversion. Each detector can be operated at three different gains of the amplifier chain. The output voltage must be reconstructed from the digitized read-out's and corrected for the gain factor.
The input to the AD converter is related to the voltage across the detector by
| VIn = VDet x G | (8.6) |
where G is the gain which can be set to one of three values. The slope in V/s is related to the output of the AD converter, in Bits/sample, by
 |
(8.7) |
The conversion factor is given by
 |
(8.8) |
The numerical values originate from the fact the A/D converter has a maximum input range of 20 V which are converted into 4095 steps, with the input voltage inverted, and that there are 24 samples per second.
Once we have calculated the slope from the digital output we can divide by the gain factor to get the detector voltage.
Values for this are held in Cal-G file 5.
This generates and fills the flag word with all relevant data.
This module converts the grating position to a grating angle, which is then used to calculate the wavelength of the light falling on the detectors. See section 7.5.
This module determines the Fabry-Pérot gap from the telemetered currents. See section 7.5.
This module calculates the wavelength of light falling on the detectors. Refer to sections 7.5 and 7.6 for a discussion of the philosophy behind this and the likely accuracy.
For the grating, this involves applying equation 7.4. The grating angle, calculated in section 8.2.12 is used along with various calibration files to determine the wavelength of light falling on each detector. Only bands with an unique order falling on their detectors are assigned a wavelength, all others are flagged as confused. It should be noted that all bands with an unique order falling on their detectors are assigned a wavelength, even if the band was not requested or contains useless data (although for non-FP observations the FP data is not transferred to the AAR).
For an AOT 01 measurement, where the grating moves during a reset interval, the grating position in the SPD is set to the grating position at the start of the reset interval. The wavelength in the SPD, however, is taken from the middle valid sample. e.g., for a one second reset interval, the grating position is that of the first sample. As samples are thrown away due to the reset pulse (see section 5.6, operation described in section 8.2.2), the wavelength is that from sample (24 - Cal-G file 3) / 2.
For the Fabry-Pérot, once the grating wavelength is calculated the mechanical gap width (calculated in section 8.2.13 is corrected to get the optical gap width (calibration file 18). The wavelength which passes through the FP is the one of which a whole multiple fits exactly in two times the optical gap width.
The data produced by the processing chain above is written out to the SPD and glitch list SWGH files. Each AOT results in one SPD file. For a definition of the SPD file see section 9.3.1. Note that the SPD file, like an ERD file, contains science data interleaved with calibration data.
A SPD record is produced for every reset interval. Complexities arise when the reset intervals for the SW and LW gratings are different. In this case there will be one SPD record (containing data for all detectors) for each of the shorter reset interval, with the SPD for the longer reset interval having its flag word set to `No data' for the records when it did not contain a reset (see table 9.8). As an example, if the SW grating had a reset interval of 1 second and the LW one of 2 seconds, a SPD record would be produced every second. Every SPD record for the SW detectors (1-24) would contain valid data, while only every other record for the LW detectors would contain valid data, the others being marked as `No data'.
