In detail, the data reduction will consist of some of the following steps:
The PHT detector output suffers from some non-linearity. Two effects
contribute: a non-linear response of the Cold Readout Electronics (CRE)
(relevant to all PHT detectors) and de-biasing effects (only for the low-biased
detectors P3, C100 and C200). A linearization correction may be
applied within PIA. It uses special calibration files and corrects for
both non-linear effects at the same time.
A cosmic radiation event ("glitch") generally results in a jump of two
consecutive read-out values, i.e. in a step within the "ramp". The signal
to be derived is therefore affected. The disturbance is usually very short
and the slope of the ramp after the glitch is similar to the slope before
it. In this case, it is not necessary to discard the whole ramp, because
PIA is able to recognise the glitches/jumps and to correct affected ramps
using the single threshold ramp deglitching procedure. This level
of deglitching is sometimes also referred to as read-out deglitching.
Ramp deglitching can be performed interactively so that the user can tune
the parameters of the procedure to optimise the performance for his/her
data. The algorithm works on a per ramp basis and therefore requires a
sufficiently large number of read-outs per ramp to be able to use a statistical
In many cases a cosmic hit produces -in addition to the disturbance
described above- a momentaneous change in the detector response, with some
relaxation time. In this case a distribution of subsequent voltage differences
between read-out pairs shows a dacaying tail, until the level before the
glitch is reached. Taking this into account an algorithm was proposed (Meyer
et Haas) based on a search for statistically significant deviant points
within the distribution of read-out pairs voltage differences using one
threshold (eg 3 sigma) and the discard of these and all subsequent points
until the corresponding voltage differences are within much more constrained
limits (eg 1 sigma) to the average of the distribution. This algorithm
takes as statistical sample a full chopper plateau or raster point (all
signals which should be the same), thus is not limited by the length of
The output of each of the integrating PHT detectors consists of the
read-out value as a function of time. A number of non-destructive integrating
read-outs (NDRs) are followed by a destructive read-out at which the integration
plateau is reset, in principle, to zero. Each sequence of NDRs defines
a ramp (voltage as a function of time); the steeper the ramp, the
higher the signal (and the power on the detector) is. Thus the signal values
are derived by fitting a 1st order polynomial to each ramp. Please
take into account that the possibility of fitting higher order polynomials
in PIA is only for calibration inverstigation purposes and should not be
performed in normal data reduction, since there is no corresponding calibration.
Chopped measurements are affected by a continuous flux change, which introduces a typical signal pattern within a chopper plateau. By the method of signal pattern determination (default for P1, P2, P3, C100 and C200 chopped measurements from PIA V8.0 on) the differential voltages between two consecutive read-outs of all the off-source chopper plateaux and all the on-source chopper plateaux are computed together, so that only two sequences (off and on) of a number N of signals in V/s is derived (by default 4 per chopper position). The temporal relative positions of the differential signals within the chopper plateaux are preserved, so that actually only a compression of all off source measured signals into one off source chopper plateau of N signals is obtained. The same with the on source positions. The characteristic temporal behaviour is preserved but with a very much reduced noise level. This makes possible a better calibration of the flux losses introduced by the continuous flux changes performed in this mode.
Additionally the user may want to have a look at the detector temperature curve which is automatically provided by PIA and which should be reasonably flat - if it is not, the signal values may be influenced by temperature fluctuations.
The user should also look at the dynamic range of the read-out values. The maximum dynamic range lies between about -0.85 and +1.09 [V] depending on the detector. Saturated read-outs will be discarded by PIA, while a poor usage of the dynamic range (e.g. -0.8 to -0.7 [V] leads to higher noise in the results. This cannot be corrected for once the measurement has been performed, but should be noted by the user.
After this processing step the data consist of the signal as a function of time.
Within this procedure the following steps are performed:
Some cosmic particle hits on a PHT detector are followed by a temporary
increase in the responsivity, thus affecting a series of read-outs and
therefore one or more signals. On a statistical basis PIA can also discard
such signals. This signal deglitching can be done interactively
so that the parameters for the algorithm which recognises affected signals
can be tuned by the user.
The detectors show transient drift/memory effects on timescales of the order of a few seconds up to a few minutes (in the worst cases). Thus the true signal values corresponding to the incoming flux are not obtained immediately, rather they are approached asymptotically after some time. Fortunately, in many cases this behaviour is a monotonic rise; however, a considerable number of pathological cases show an overshoot followed by a slow decay.
To account properly for detector drifts is one of the most difficult tasks of the data reduction. PIA provides different ways of handling this problem:
For every chopper plateau in a measurement averages and medians of the
corresponding signals are calculated. The results are combined with other
reduced variables in the signal-per-chopper-plateau (SCP) buffer.
Instead of calculating a simple average per compressed chopper plateau
(one on- and one off-source) the signal difference between mean on- and
off-values (different determination depending on the detector used) is
used for computing a signal difference, corrected according to calibration
tables. This difference is used for recompute a mean on and a mean off
signal which is passed to the SCP level. Please note that two corrections
must be performed before: the reset interval effect correction and the
dark current subtraction. (Both can be performed on the SRD and on the
SCP level by PIA, we recommend however to apply them on the SCP level by
staring measurements or by chopped measurements which are processed in
the traditional way [not by the "signal pattern" recognition]. They are
described under the SCP corrections)
Since the actual responsivity should usually be used for the power derivation,
we describe the processing of FCS calibration measurements first, including
the data corrections which can be applied.
It has been found that the signals obtained from the CRE output are
partly dependent on the used reset interval (ramp integration time) for
sampling. This correction normalizes to the output one would have obtained
if measuring with an integration time of 1/4 of a second. This correction
must be applied (since PIA V7.0) because the whole calibration has now
been determined accordingly. Although it can be also applied on the SRD
level, we recommend to apply it on the SCP level, prior to the dark current
subtraction (except for the case of "signal pattern" analysis by chopped
The signal per ramp contains a more or less constant component coming from the dark current in the detectors. Dark current values per pixel and detector are contained in calibration files. They can be used for subtracting this component from every measurement. This correction is also more efficiently applied here than to the former level of reduction (SRD).
Since PIA V7.0 the dependence of the dark current on the orbital position
of ISO is included within the calibration data and the corresponding algorithms.
Several interpolations are possible, they are based on one value per signal,
per chopper plateau, per raster or per AOT.
A dependence of the detectors response on the illumination level has been established. Since all measurements are affected by this effect (also the ones used for establishing the Responsivity, the FCS measurements) the linearization of the detector response is better handled by a linearization of all signals, as recorded by the detectors. This correction can be applied using PIA also both on the SRD and the SCP level, but except for chopped measurements reduced using the "ramp pattern" procedure it should be better performed on the SCP level.
When the optical path to the detector is directed to one of the internal
Fine Calibration Sources (FCSs), the telescope field of view is pointing
to the sky. This yields a certain amount of straylight on the detector,
which affects the signal measured from the FCS source. It is very difficult
to correct for this effect. Nevertheless, PIA offers a way to at least
estimate the influence of such an undesired effect on the responsivity
calculation. This correction is optional; it is most important when the
sky is fairly bright.
The responsivity of the detector(s) is derived from the FCS calibration measurement. It is assumed that the responsivity from one FCS measurement can be applied to all the data in a given AOT, which is a reasonable assumption if the AOT is not too long and the flux changes within the AOT are not too drastic. For the responsivity computation, the mean signals of the FCS measurements are compared to the (in principle) known optical power on the detector corresponding to the electrical power applied to the FCS. The relationship between the optical power and the electrical power is contained in calibration files.
When the chopper throw is large, all PHT detectors can be telescope-
vignetted. In the case of the PHT-P detectors the effect depends on the
aperture used. The correction for this effect is taken from calibration
"Internal" background subtraction refers to the calculation of the source signal from an on-source/off-source chopped measurement. This can also be done after the data have been calibrated to in-band power.
The result of a background subtraction is a single value (e.g. mean source - background), which can be written into the SCP buffer as a subtracted pseudo-measurement with a single chopper plateau. This buffer must be selected for processing in order to use the subtracted data for further reduction.
Selection/de-selection of chopper plateaux is possible within PIA and can be used to control what data are included in the source-background calculation. Editing of the data at this level may be useful when anomalies affect one or several chopper plateaux; this can usually be determined by visual inspection of the (non-background-subtracted) SCP data.
In most cases external background subtraction (using a separate
background measurement) should be done at the SPD level since the responsivity
may have changed between execution of the source and reference observation.
However, it is possible to do the subtraction on data reduced only to SCP
level, which may be desirable in cases where the same (e.g. default) responsivity
must be used to derive the optical power for both source and reference
observations. Since external background subtraction is normally performed
on data reduced to the SPD level, this is described in detail below.
[This correction which should still be applied to chopped measurements which have not been "signal pattern" reduced, is now obsolete in the default processing, since the calibration of the signal pattern includes already the correction]
The subtracted value from an on-source/off-source chopped measurement
depends in the first approximation on the chopping frequency used (and
in a second order on the signal difference between both levels). Empirical
values have been determined which correct for these signal losses. One
should only make use of these correction factors in the case of subtracted
chopped measurements (PIA enables the correction only in this case) and
if no drift modelling has been applied to the data.
By very long PHT measurements a baseline drift is often observed, which
can be related to slow changes in the detector responsivity. While signal
drift modelling on a short timescale can be performed using diverse functions
(see transients modelling), there is no
easy approach for long term drifts . In some cases, eg mapping using the
P32 mode, it is impossible due to the superimposed signal changes by chopping
to diverse positions. A very user interactive approach was chosen for correcting
in these cases, based on the judgement by eye-balling. The user has the
possibility of defining several points within a measurement which should
have a similar flux but show different signals. From these points factors
are derived for correcting the measurement, interpolating them for every
point in the chosen distribution.
To convert signal per chopper plateau in V/s to in-band power in W it is necessary to know the responsivity of the detector, which varies during a revolution. There are two possibilities of which responsivity to use when doing the calculation: the default detector responsivities as contained in general calibration files, or the actual responsivities, as calculated from the associated FCS measurement and described above. While this second alternative should always be better, default responsivities may need to be used in special cases, such as failed FCS measurements. In the case of PHT-S default responsivities are the only choice, since there are no FCS measurements incorporated into the observations.
The detector response varies for certain detectors with a clear tendency
within an orbit due to the continuous radiation to which the detector is
exposed. This fact is reflected in orbital position dependent
default responsivities, which have been established and are used from
PIA V7.0 on. A value per measurement or per AOT can be computed by interpolating
within the numbers present in the calibration files.
From PIA V8.0 on, the power calibration leads directly to the AAP level, thus contain the optical corrections described under the next point. Nevertheless, SPD level data is produced and can be inspected by using the dedicated GUIs. Although it would not be necessary at all by pure PIA data processing, we keep in this way compatibility to the PHT OLP pipeline, which only knows SPD as intermediate data processing stage.
A background measurement from another observation can be subtracted
from a source measurement. The result of this subtraction can be stored
in the SPD buffer as a subtracted (pseudo-) measurement for further reduction.
This subtracted measurement must be selected for processing if it is to
be used for subsequent data reduction.
When computing Jy or MJy/sr, PIA applies an optical correction to the in-band power (in W). This optical correction takes into account the known properties of filter, aperture and other optical elements (stored in Calibration Files).
Correction for pointing offset by PHT-S observations:
Due to the small pixel size of PHT-S (24"x24") and to the sharply peaked footprint shape, already a few arcsecond off-centre positioning may lead to both significant intensity changes as well as to spectral shape alterations. Footprint matrices for every PHT-S pixel can be used for correcting depending on the established offset (see section 3.3.10. Pointing correction for PHT-S).
It is at this stage that the specific data reduction according to the final observation objectives takes place. PIA offers special data reduction modes for every type of observation, as listed below:
The results from different measurements and detector pixels can be combined,
yielding photometric tables and plots of flux density/surface brightness
versus wavelength, depending on the choice of point or extended source
photometry. The possibility of performing colour corrections is included,
for which the user has a choice among a black-body spectrum of a user's
chosen temperature or a power-law spectrum of a user's chosen index.
Measurements using the same filter but different apertures can be combined,
yielding curves of growth (in Jy and MJy/sr), both as tables and plots.
In addition, colour corrections are possible here, as they were described
for multi-filter photometry.
A PHT-S spectrum of flux density/surface brighntess per raster point
and chopper plateau can be analysed by fitting lines and continuum. Fit
parameters can be varied and regions of the spectrum defined. Plots and
tables of the spectra can be also obtained.
Maps in RA/DEC can be made from the surface brightnesses (or fluxes)
and corresponding astronomical coordinates of a mapping measurement. PIA
allows flexibility in mapping procedures and parameters. E.g. the pixel
size of the map, detector pixel selection, automatic flat fielding, two-
and three-dimensional map display, profiles, map zooming, colour table
selection, etc. can be varied.
Measurements using different polarizers can be combined. Polarimetry tables and plots can be obtained. In addition, colour corrections are possible here, as they were described for multi-filter photometry.
|13/05/1996||Martin Haas (MPIA) / Carlos Gabriel (ESA-SAI)||First Version|
|19/07/1996||Carlos Gabriel (ESA-SAI)||Revised|
|26/02/1997||Carlos Gabriel (ESA-SAI)||Updated|
|03/06/1997||Carlos Gabriel (ESA-SAI)||Update (V6.3)|
|09/07/1997||Carlos Gabriel (ESA-SAI)||Update (V6.4)|
|12/02/1998||Carlos Gabriel (ESA-SAI)||Update (V7.0)|
|15/08/1999||Carlos Gabriel (ESA-SAI)||Update (V8.0)|
|02/11/1999||Carlos Gabriel (ESA-SAI)||Update (V8.1)|