6.) Advanced PIA use - working interactively on IDL with PIA structures and programmes

The PHT Interactive Analysis offers the possibility of using all the package routines and the buffers fully interactively (from the IDL prompt). Thus, the information contained in all buffers can be extracted, visualized, changed, etc. It is also possible to leave and re-enter PIA after having done some data reduction, using the newly obtained data. Of course, this last point is only possible as long as the internal buffers are not corrupted, and the data re-entered in the PIA buffers entirely respect the PIA structures definitions. From PIA V7.0 a new IDL feature is used, which allows to access fully interactively all buffers and routines, while the GUI manager is running.

The advantages of such a possibility are obvious. A graphical I/F can never cope with all the special desires and requirements which an astronomer can have for special data reduction (even such a clever and sophisticated I/F has its limits!)

In this chapter we mainly describe the ways of extracting the data from the buffers (dynamical structures) into real structures to be used on the IDL main level and ways of re-incorporating data into PIA, in order to make use of the full graphical I/F after having modified/reduced data. An example is also given for better comprehension.

While PIA is working, you remain in the IDL general session

It is not necessary anymore to exit PIA, closing the PIA windows in order to access the buffers from the IDL prompt.

All the buffers and compiled functions and routines are in the IDL memory. They have just to be made known to the IDL environment (the IDL main session).

Loading the PIA buffers

To load the common blocks containing the PIA buffers to the IDL main session, please enter at the IDL prompt:

IDL> @pia_on_line

In this way a script is included (through the @) which declares via include symbols the various common block declarations containing the dynamical PIA buffers. As explained in Appendix A: Data Structures and Files they contain the handles (IDL pointers) addressing the data.

Now the common blocks phterd, phtsrd, phtscp, phtspd and phtaap are known to your environment. They contain the measurements which have been read/processed in the different levels, 'piled up' within these buffers. To see how many measurements are contained e.g. on the phtsrd level you just type:

idl> help, phtsrd

The answer will be something like: PHTSRD (PHTSRD) STRUCT =-> PHTSRD_TYPE Array(2) indicating that phtsrd is a common block containining one structure of several structures of phtsrd_type (an IDL structure is a collection of scalars, arrays or other structures contained in a variable). Two elements are present in our example, the first element (0th in IDL convention) always being a dummy, used for initialization. Thus, in this case, only one measurement is present in the buffer.

To obtain information on phtsrd type: idl> pia_structures_help, /srd; a list of the variables contained within phtsrd will be given (the same applies for phterd giving /erd as the keyword, /scp for phtscp, /spd for phtspd and /aap for phtaap).

You can now access the contents of the structure corresponding to the valid measurement (phtsrd(1)) through different commands, depending on whether or not the element in the structure is directly a variable or a pointer. To avoid this complication and to gain more transparency in the operations, PIA offers the possibility of transforming every one of the structures within the dynamic array into a 'real' structure.

Converting dynamical structures elements to 'real' structures

A 'real' structure is a structure of all types of variables, except pointers. It contains the same number of variables as those in the buffers, but with data extracted from the pointers into 'normal' variables. All the variables also preserve their names. The advantage is that the variables can be accessed and manipulated directly. A call of a conversion routine like

idl> pia_load_isrd, phtsrd(1), tmp

takes all the variables corresponding to the first measurement in the phtsrd common block, despite being normal variables or pointers, and accomodate them into a structure of normal variables. Input (phtsrd(1)) refers to the second element in the two elements structure array, output is an anonymous structure named tmp in this case (you can of course choose the name). Similar routines exist for all levels, the convention is pia_load_iXXX, where XXX stays for the data level (erd, srd, scp, spd, aap).

A bit (or more) of IDL knowledge is now necessary to work with the data contained in these structures. Please refer to the IDL standard help and manuals for this purpose.

The PIA routines can also be used, since practically all of them (except for the specific graphical I/Fs for the various levels, and the general menus) work using 'real' structures (although sometimes the results go into the dynamical buffers). For this help pages on each of the ~ 500 files containing one or more routines are written, which can be accessed by the PIA source button appearing in the IDL help window, started by:

idl> widget_olh

(NOTE: Some of the information contained in these help pages are to be completed/corrected. )

PIA provides you with the means of re-entering temporary structures within the buffers, e.g. after changes or data manipulations.

Re-importing 'real' structures into the PIA buffers

First a warning: you have to be careful when importing data into the PIA buffers, since this can lead to problems, in cases of anomalous structures, like structures with arrays of different lengths, empty variables, variables with a different type specification as expected, etc. Please check the formats of your 'structures' as far as you can before accomodating them into the PIA buffers.

A temporary structure tmpscp of scp type can be added to the end of the phtscp buffer by:

idl> pia_in_buffer, tmpscp

The name of the structure is not relevant for the target buffer, but parts of the input structure are analysed to decide which buffer is the corresponding one. This is done to prevent users from putting data into wrong buffers. If you want to accomodate the structure in a buffer position already occupied by a measurement (e.g. for replacing data by changed data), the call is

idl> pia_in_buffer, tmpscp, replace=n

with n being the position chosen. Of course this position has to exist. If it does not, an error message will be returned. The data stored in the chosen position is replaced by the data corresponding to the input structure.

Help to identify the measurements can be obtained using the decode_admn routine, e.g.

idl> decode_admn, phtscp(3).admn, meas

idl> print, meas

The output variable meas is a string variable giving detector, TDT and measurement number corresponding to the measurement stored under phtscp(3). The same routine can be used for the 'real' structure, just replacing phtscp(3) by tmpscp.admn in the call.

Using scripts for data reduction

Another advanced way of using PIA outside the graphical I/F is through one's own programmes, which perform one or more steps, combining PIA buffers, PIA routines and other IDL routines. A good example of this is given in the file PIA_script_example.pro, which is included in the support directory under the main PIA directory. This programme is the one corresponding to the automatic general processing given under sequences in the main PIA widget. Special care has been taken to document it internally for easy comprehension. Since it covers the full data reduction including all correction possibilities, it surely represents a good aid to understanding and reproducing parts of it.

Another possibility is as follows: replacing calls to PIA modules within this script with calls to one's own modules is also a choice, and this would represent a major step towards extending the PIA capabilities to one's own needs.

In addition, and for faster access to the individual help modules, in the chapter on Processing Algorithms, you will find the name of each of the main routines performing the tasks described there.