Conversion – Functions

While providing functionality similar capabilities for functions calls and dynamic parameter lists, the IDL and Python implementation patterns are quite different. IDL provides a “functional” based approach, tuned to the needs of the interactive user and based on it’s FORTRAN roots. On the other hand, Pythons implementation is based on modern methodologies, based on common vararg patterns and leveraging it’s built in data structures.

The major functional arguments differences between IDL and Python include:

  • Python only supports functions and not subroutines.
  • IDL uses functions and special keyword names to manage dynamic parameter lists, while Python uses standard list data types.
  • No partial keyword expansion capability in Python – full keyword symbols are required when making routine calls.
  • Input parameters and keywords are passed by value, meaning output values are not supported by function parameters. Output values are only supported as the return value from a Python function. Unlike IDL, Python functions can return more than one value though.
  • In Python, undefined variables in a function call is an error. In IDL, the use of undefined variables is a key implementation pattern.

While the function structure and syntax gaps between IDL and Python are large, Pike addresses each through a straight syntax translation and runtime functionality.

Function Translation

Pike takes several approaches to translating an IDL function into Python, providing either a static or dynamic implementation pattern.

The static functional translation pattern provides a standard Python function signature that almost identical to IDLs. Pike uses this translation method when the routine being translated meets the following criteria:

  • Doesn’t make use of dynamic parameters (doesn’t call n_params())
  • No output parameters or output keywords are supported.
  • The function doesn’t support the _EXTRA keyword
  • The ‘always_varargs’ preference is false.

If this  criteria is met, Pike writes a standard, static function signature for the routine being converted. Otherwise the dynamic function signature pattern is used during the conversion.

IDL Routine Definition

To determine what method to use during the translation process, as well as how to call a translated routine, Pike requires a detailed definition of each routine signature. This definition is a key element to Pikes effectiveness and must included the following information:

  • The routines name
  • The name of the file that contains the routine
  • If the routine is a function or procedure
  • Which parameters provide output values
  • Which keywords provide output values
  • Full keyword names

Some of this information is easily obtained from the IDL source code, while some, like output values is not. To determine this information, Pike implements a multi-phased approach that includes the following:

  • Determine the routine name, parameters and keywords during the indexing phase of the translation process.
  • If a routine contains IDL style comments, parse the comment block, scanning for output parameters and keywords.  This is done during the indexing phase of the translation process.
  • During the optimization phase of the translation process, Identify undefined variables in function calls and mark those parameters as output values.
  • Manual entry of a routines definition in the Pike library file that supports the routine being translated.

Once this information is determined, Pike stores the routines definition in a Pike library file (__pikelib__.pkl) in the same directory as the file being translated. Pike (pikex) uses this library file to properly generate the function calls to the translated IDL routines.

Dynamic Function Capabilities

To support the dynamic parameter capabilities of IDL in Python, Pike creates an object that manages a routines input parameters, output parameters and the set of unique IDL entities that support dynamic parameters.

In this method, Pike writes the translated function signature following the standard Python variable argument lists pattern: an argument lists (args) and a keyword dictionary (argk).

      myroutine(args, argk):

Pike then adds a call to the Pike specific routine routineParameters(), providing the following information as arguments to this method call:

  • The pass in Python parameter list (args) and keywords dictionary (argk)
  • Parameter names
  • Keywords names and the local variables used by the keywords
  • Identifies the output parameters and output keywords

This Pike system function returns an object, which is used to manage the input and output arguments within the routine.

And example of a function translation in Pike:

In IDL:

Where the 2nd and 4th parameter, and kwone are output arguments.

function myfunction, one, two, three, four, kwone=kw1, kwtwo=kw2

In Python:

      def t_outargs_func(*args, **argk):
             __args = pike.routineParameters(args, argk, 
                            parameters=['one', 'two', 'three', 'four'],
                            keywords={"kwone":"kw1", "kwtwo":"kw2"}, outparams=[2, 4],
                            outkeys=['kwone'])

Parameters and Local Keywords

The passed in parameters and keyword values are accessed as attributes (properties) of the created Pike dynamic parameter object (__args in the above example). This preserves the variable names of the original IDL routine, while allowing Pike to manage output parameter values.

In IDL:

      if(n_elements(kw1))then $
          kw1 = kw1 + 1

In Python:

    if (pike.n_elements(__args.kw1)):    
        __args.kw1 = __args.kw1 + 1

_EXTRA Keyword Support

IDL supports dynamic keyword argument support using the special keyword name _EXTRA, which IDL maps to a structure where each field maps to an unhanded keyword passed in to the routine.

Pike provides the same functionality, creating a structure (a Pike library implementation of an IDL structure) where each unassigned keyword is mapped to a field, and assigning that structure to the local _EXTRA variable exposed as an attribute on the dynamic parameter object.

The following example, the keyword “cow” is used when calling the function t_keyword_extra.

In IDL:

      function t_keyword_extra, cow, pig, _extra=_eE
	   if( _EE.coW ne cow)then $
    		return, 1

In Python:

    def t_keyword_extra(*args, **argk):

   	 __args = pike.routineParameters(args, argk, parameters=['cow', 'pig'], 
    			keywords={"_extra":"_eE"}, outparams=[], outkeys=[]) 
    
  	 if (__args._eE.cow != __args.cow):    
        	return __args.returnValues(1)

Return Values

As noted earlier, for Python, all output values from a function are returned as the value from a function. Python supports the ability to return multiple values from a function through the use of its builtin tuple data type.

For IDL, all routine parameters and keywords support output value functionality. As such, Pike must identify and add output values to the return statements of the Python implementation. Pike does this using the capabilities of the aforementioned dynamic argument object.

The Pike dynamic argument object implements a method that appends all output parameters to the return values of function. Written as part of the pikex translation output, this method, returnValues(), is passed in the original return value of a function, if one exists, where it appends all output values for the routine and returns a Python tuple value which is returned from the function.

The translated call to this function is written such to support multiple return values from the routine.

The following example shows how and IDL function with multiple output parameters:

In IDL:

      function t_outargs_func, one, two, three, four, kwone=kw1, kwtwo=kw2
           ;; ... routine logic, with a return value of 0
	   return, 0

     ;; And the function is called like:
	result = t_outargs_func(p1, p2, p3, p4, kwtwo=3)

In Python:

    def t_outargs_func(*args, **argk):

    	_args = pike.routineParameters(args, argk, parameters=['one', 'two', 'three', 'four'], 
    			keywords={"kwone":"kw1", "kwtwo":"kw2"}, outparams=[2, 4], 		
			outkeys=['kwone']) 
        #  ... routine logic, with a return value of 0
	return __args.returnValues(0)

    # And the function is called like (note the multiple return values):
      result, p2, p4 = t_outargs_func(p1, p2, p3, p4, kwtwo=3)

Inline Functions

In Python, if a function supports multiple return values, the functions output cannot be used as input to another routine or in a control statement. To manage this in the translation process, Pike will pull the function from it’s original use, assign the output value to a temporary variable, and use this temporary value as input.

In IDL:

    if( t_outargs_func(kwtwo=44, kwone=key1) ne 1)then begin

In Python:

    __tmp0, key1 = t_outargs_func(kwone=key1, kwtwo=44)
    	if (__tmp0 != 1):