Home
About ISIS
Support
Download

ISIS 3

Documentation
Tutorials
Technical Documents

ISIS 2

Documentation
Tutorials
Technical Documents

Search

USGS

ISIS 3 Application Documentation


hicalbeta

Printer Friendly View | TOC | Home

Performs radiometric calibration of HiRISE channel images

Overview Parameters

Description

hicalbeta appiles radiometric calibration correction to HiRISE images. This particular version is deemed experimental, thus the beta designation. The radiometric calibration correction is performed on each individual HiRISE channel file (EDR) correcting for drift, instrument offset, dark current, line-dependant gain, gain, flat field and finally, conversion to I/F, DN/uS or DNs. There are 14 different CCDs with 2 channels each for a total of 28 channels. (Note that channels are the basic HiRISE image product where calibration is applied.)

Some of the calibration data are contained within each ISIS HiRISE cube image as ancillary data. These data are stored in ISIS BLOBs (seen in the labels as Table objects). There are three primary BLOB Tables used in the calibration:

  • HiRISE Calibration Image - Contains calibration image data. These data are acquired immediately before the actual image observation from three distinct calibration sources. Reverse Clock contains 20 lines of nominal zero (offset) data. There is some serial register dark current but this is negligible at the shorter line times. Mask contains 20/bin lines of data from behind the aluminum mask. It should only contain dark current - serial and parallel. Ramp is 8, 32, 64 or 128 time delay integration (TDI) lines that includes signal from the reverse and forward clocking. For a uniform scene, the peak amplitude of the ramp should be 2X the image area DN.
  • HiRISE Calibration Ancillary - These data are also acquired prior to the actual image. This BLOB contains two types of calibration data. Buffer Pixels are the 12 pixels at the start of each line of the calibration image. Dark Pixels are the 16 pixels at the end of each line under the aluminum mask.
  • HiRISE Ancillary - These data are acquired for each image line. This BLOB contains the same data as the HiRISE Calibration Ancillary; the Buffer Pixels and Dark Pixels data contained here are for each image line and will equal the total number of image lines.
Note that these data BLOBs are removed by hicalbeta in the output file so that repeated runs of this application are prevented.

The calibration is carried out in a series of modules. These modules provide various contributions to the calibration process. The parameters that govern the behavior of these modules are contained in the program configuration file as specified in the CONF program parameter. The default provided is automatically selected from the ISIS MRO ancillary data that accompany the ISIS release and would normally not need to be explicitly provided by the user.

Each of the calibration modules has their own unique set of keywords that govern is behavior. This is documented in the CONF parameter.

The equation that is applied is described below:

 Module Processing Summary:

          ZeroBufferSmooth          - Fills gaps in Buffer Data (ZBS)
          ZeroBufferFit             - Computes non-linear fit of ZeroBufferSmooth (ZBF)
          ZeroReverse               - Process Reverse Clocked data (ZR)
          ZeroDark                  - Dark current temperature correction (ZD) 
          GainLineDrift             - Time-base line drift correction (GLD)
          GainChannelNormalize      - Normalize gain for summing/TDI (GCN)
          GainNonLinearity          - Line-based non-linearity gain correction (GNL)
          GainFlatField             - Flat field correction (GFF)
          GainTemperature           - Temperature dependant gain correction (GT) 
          GainUnitConversion        - DN unit conversion factors (GUC) 

  The general form of the HiRISE calibration equation is:

    oDN = (iDN - ZBF(ZBS) - ZR - ZD)  / GLD * GCN * GNL * GFF * GT / GUC 

          where iDN is the input raw pixel value and oDN is the calibrated 
          output pixel value in 16-bit DN.
      

Computation of these radiometric components is governed by the PVL ()CONF) configuration file. This file is highly customizable to accommodate as many specialized processing needs as possible. The contents of this file and how it may be used/customized is fully described below in this documentation section.

This important file provides all parameters used in the calibration process. It contains references to calibration matrices (such as flat fields, instrument gains, etc...), label keywords and parameters used in the radiometric calibration process.

This highly flexible configuration file contains numerous module profiles that govern the behavior of each phase in the calibration process. These modules each have parameters that are specific to their function. There are 10 different modules used in the application. However, the configuration file will typically include many other profiles. The user can take advantage of the ability to merge other profiles into one or more of the other main profiles, simply by the content of the labels or observing conditions of the image.

The file must contain a top level Hical object. Within this object are numerous profiles. The keywords that are contained in the Object section of the file are always used in every profile and can be superseded by any subsequent profile loaded after the initial one. Optional profile names are constructed by the combination of the OptionKeywords and ProfileOptions keywords. The OptionKeywords lists keyword groups in the configuration file and/or label that can be used to textually replace the patterns surrounded by the {} in the ProfileOptions keyword. The keywords FILTER, CCD, CHANNEL, TDI and BIN are always generated after the initial module and label are loaded in. After the initial profile is loaded, the FROM file label is loaded using the LabelGroups list to determine which keywords are loaded. Note that the groups must exist or an error is issued. Otherwise, the user can specify any defined keyword in the OptionKeywords list to apply to profile names.

The real power of the configuration file is its use of named Profiles. Profiles are groups of keywords that can be associated to a unique definition. The hicalbeta PVL configuration file consists of a single Hical object with numerous named Profile groups. Each of the Profile groups must contain a Name keyword that uniquely identifies it within the Hical object. This allows us to create Profiles that pertain to particular combinations of filter (RED, IR, BG), CCD (RED0-9, IR10/11, or BG12/13), TDI (128, 64, 32, 8), BIN (1, 2, 3, 4, 8, 16) or CHANNEL (0 or 1). These values are determined from the content of the HiRISE label. Combinations of profiles that are added after the initial default are specified in the ProfileOptions keyword. Profile combinations can consist of any combination or use of these defined values. These defined values are specified as or with the Name keyword in Profile groups delimited by curly braces. Given this definition one can specify a particular group of calibration parameters for a specific CCD channel with the pattern {FILTER}{CCD}_{CHANNEL}. Then, one can define a special collection of calibration matrices, keywords or scalars for any one (or none) of the filters. So, for the problematic IR10, you can have a named profile called IR10_1 whose keywords in the profile are loaded when calibrating a IR10_1 channel, thus overriding any defaulted keywords in all profiles. Should named profiles using this option not exist, they are ignored. Also, profiles specified in this manner are loaded in the order specified in the ProfileOptions keyword, thus creating a hierarchy of calibration specification configurations.

OptionKeywords specify replacement patterns that exist in retrieval of all filename references in the configuration file. This includes virtually all files including matrix and CSV files.

Matrices are comma separated values (CSV) files that contain rows and columns as content requires. There is expected content in the CSV matrix file to support 28 HiRISE CCD channels where indicated. Module requirements dictate matrix content, however. There will always be one line for these CSV files since HiRISE detectors are line scan instruments. There will be a minimum of 64 (for summing mode of 16) and a maximum of 1024 (summing mode of 1) samples in these files. The content of these matrix files may also be dependent upon summing mode and the time delay integration (TDI) mode used during image acquisition. TDI allows varying number of line scans, in conjunction with exposure time, to pass over the same point on the surface of Mars to increase the signal-to-noise ration (SNR) as well as resolution. There are 4 selectable modes of TDI: 128, 64, 32 and 8. Coupled with 6 different summing modes (1, 2, 3, 4, 8, and 16), there are at most 24 calibration matrices per set. These matrices are referenced as file paths and patterns of the form:

 Flats = $mro/calibration/matrices/A_TDI{TDI}_BIN{BIN}_????.csv
                
Here, {TDI} and {BIN} correspond to the TDI and summing mode used during image acquisition, respectively. Matrix variables are equated to file references using the pattern substituion of keywords enclosed in {} in the configuration file. Matrix file paths are also subjected to pattern replacement in the same fashion as profiles. This will minimize the content management aspect of the configuration file and encourage consistant file naming schemes.

Below is an example of a complete PVL configuration file that demonstrates some of the features described:


#  HiRISE Calibration Matricies configuration file
#  See documentation for the hicalbeta application on the content and form of
#  this file.
Object = Hical

  Program = "hicalbeta"

  Name           = "HiMatrices"
  DefaultProfile = "HiMatrices"

/* If you want to rerun hicalbeta, you must set PropagateTables to True.  Use */
/* this in conjuction with Debug::SkipModule = True option for each module. */
  PropagateTables = False

/* Define label groups that are loaded for each profile reference */
/* Note all keywords in these groups become available to all profiles. */
/* Thus, you can use them in the OptionKeywords and ProfileOptions keywords */
/* to create very specialized profiles for special needs. */
/* Specify the FPA reference temperature.  It is used in several modules so */
/* it is specified at the top level */

  LabelGroups = ( "Dimensions", "Instrument", "Archive")

/* These keywords are used in ProfileOptions mapping.  Note that order and */
/* case matter!  WARNING:  You can easily break file lookups if these keys */
/* are deleted or modified improperly!!! */
  OptionKeywords = ("FILTER", "CCD", "CHANNEL", "TDI", "BIN", "ProductId",
                    "Program", "Module", "OPATH", "CalOptions")

/* Additional profile combinations and order load hierarchy.  These keywords */
/* are defined when the LabelGroups are loaded. */ 
/* Kris Becker & Eric Eliason updated 10/24/2008 */
/* ProfileOptions value added: {Module}_{CalOptions} */
  ProfileOptions = ("{FILTER}", "TDI{TDI}", "BIN{BIN}", "TDI{TDI}/BIN{BIN}",
                    "{FILTER}{CCD}_{CHANNEL}",
                    "{FILTER}{CCD}_{CHANNEL}/TDI{TDI}/BIN{BIN}", "Debug",
                    "{Module}_{CalOptions}")

/* Specify the FPA reference temperature.  It is used in several modules so */
/* it is specified at the top level */
  FpaReferenceTemperature = 21.0

  /* This profile contains parameters pertinent to processing the buffer */
  /* pixels for subsequent using in the down-line offset correction, ZeroBufferFit module */
  Group = Profile
    Name = ZeroBufferSmooth
    Module = ZeroBufferSmooth
    ZeroBufferSmoothFirstSample = 5
    ZeroBufferSmoothLastSample = 11

    ZeroBufferSmoothFilterWidth = 201
    ZeroBufferSmoothFilterIterations = 2
  End_Group

/* This profile contains parameters pertinent to processing the down-line offset correction */
/* We do not use this by default because a functional fit misses important bumps and wiggles in */
/* the buffer pixels, which are required to remove similar noise from the image area. */

  Group = Profile
    Name = ZeroBufferFit
    Module = ZeroBufferFit

  /* Uncomment to turn off non linear fitting of ZeroBufferSmooth data and pass it thru */
    ZeroBufferFitSkipFit = True

  /* Uncomment to use linear fitting of ZeroBufferSmooth data when non-linear fails */
  /* Default is to pass thru filtered ZeroBufferSmooth data */
  /* ZeroBufferFitOnFailUseLinear = True */

  /* Minimum number of good lines (NLines - (TrimLines/Summing)) to fit */
    ZeroBufferFitMinimumLines = 250

  /* Maximum  number of iterations for the algorithm to converge and */
  /* other limits */
    MaximumIterations = 100
    MaximumLog        = 709.0

  /*  Convergence parameters for Levenberg-Marquardt algorithm */
  /*  Equation is solved when |dx_i| < AbsoluteError + RelativeError * |x_i| */
  /*    where dx is the last step and x is the current step for each i-th */
  /*    value */
    AbsoluteError = 1.0E-4
    RelativeError = 1.0E-4

  /* Filtering of the guestimate buffer */
    GuessFilterWidth = 17
    GuessFilterIterations = 1

  #    DumpModuleFile = "{ProductId}_{Module}.log"
  End_Group

/* This profile contains parameters pertinent to processing the offset in the columns */
  Group = Profile
    Name = ZeroReverse
    Module = ZeroReverse

/* Set calibration parameters for hiclean operations.  Indexes are all 0-based */
    ZeroReverseFirstLine = 1
    ZeroReverseLastLine  = 19


/* Reverse Clock trigger Statistics profiles */
    ReverseClockStatistics = "$mro/calibration/matrices/beta/ReverseClockStatistics.????.conf"
    RevLisTolerance = 1
    RevHisTolerance = 1
    RevNulTolerance = 1
  End_Group

/*  Skip reverse clock if the ReverseReadoutNoise is to large */
/*  Profile added by Kris Becker & Eric Eliason, 10/25/2008 */
  Group = Profile
    Name = Zz_ReverseReadoutNoise
    Debug::SkipModule = True
  End_Group

/* This profile contains parameters pertinent to processing dark current. */
/* Required label keywords:  Summing, Tdi, FpaPositiveYTemperature, */
/*                           and FpaNegativeYTemperature, Lines */
/* Also needs LineTime which is computed. */
  Group = Profile
    Name = ZeroDark
    Module = ZeroDark

/* Define the B matrix file reference */
    B = "$mro/calibration/matrices/beta/B_TDI{TDI}_BIN{BIN}_hicalbeta_????.cub"
    SkipLines        = 1
    Slope            = "$mro/calibration/matrices/beta/t_slope_CH{CHANNEL}_hicalbeta_????.csv"
    Intercept        = "$mro/calibration/matrices/beta/t_intercept_CH{CHANNEL}_hicalbeta_????.csv"

/* As of version 0020, 9 March 2010, we will be using the following names and formats */
/* We are calling this the DarkCurrent now.  The filename will stay the same (with a .csv extension) */
    DarkCurrent = "$mro/calibration/matrices/beta/B_TDI{TDI}_BIN{BIN}_hicalbeta_????.csv"
    DarkCurrentColumnName = "{CCD}/{CHANNEL}"

/* The slope and intercepts to the temperature-dependent correction to the dark current are given below. */
    DarkSlope = "$mro/calibration/matrices/beta/B_Temperature_Slope_hicalbeta_????.csv"
    DarkSlopeColumnName = "CH{CHANNEL}_TDI{TDI}"

    DarkIntercept = "$mro/calibration/matrices/beta/B_Temperature_Intercept_hicalbeta_????.csv"
    DarkInterceptColumnName = "CH{CHANNEL}_TDI{TDI}"

    /* Do filtering? */
    ZeroDarkFilterWidth      = 3
    ZeroDarkFilterIterations = 1
  End_Group

/* This profile contains parameters pertinent to processing the nonlinear gain correction */
  Group = Profile
    Name = GainNonLinearity
    Module = GainNonLinearity
  
/* Define the nonlinearity correction coefficients */
    NonLinearityGain = "$mro/calibration/matrices/beta/Gain_NonLinearity_BIN{BIN}_hicalbeta_????.csv"
    NonLinearityGainRowName  = "{CCD}_{CHANNEL}"
  End_Group



/* This profile contains parameters pertinent to processing down-line gain correction. */
/* Required label keywords: CpmmNumber, ChannelNumber, Lines */
/* This module has been reactivated as of 2010-01-29 */
  Group = Profile
    Name = GainLineDrift
    Module = GainLineDrift

    SkipLines            = 1
/* Added "_hicalbeta" to filename as of 2008-04-02.  This is consistant with */
/* naming convention used for the beta version of hical. */
    GainLineCoefficients = "$mro/calibration/matrices/beta/line_correct_{BIN}_hicalbeta_????.csv"

/* As of version 0020, 9 March 2010, these are more correctly, called the LineGainDrift correction*/
    LineGainDrift = "$mro/calibration/matrices/beta/Line_Gain_Drift_BIN{BIN}_hicalbeta_????.csv"
    LineGainDriftColumnHeader = True
    LineGainDriftRowName = "{CCD}/{CHANNEL}"
  End_Group

/* This profile contains parameters pertinent to processing gain correction. */
/* Required label keywords: CpmmNumber, ChannelNumber, Lines */
  Group = Profile
    Name = GainChannelNormalize
    Module = GainChannelNormalize

/* Define the G matrix file reference */
    G = "$mro/calibration/matrices/beta/G_TDI{TDI}_BIN{BIN}_hicalbeta_????.cub"

/* As of version 0020, 9 March 2010, these coefficients have been renamed to Gains. */
    Gains = "$mro/calibration/matrices/beta/Gains_hicalbeta_????.csv"
    GainsRowName = "{BIN}"
    GainsColumnName = "{CCD}/{CHANNEL}"
  End_Group

/* This profile contains parameters pertinent to processing gain correction. */
/* Required label keywords: CpmmNumber, ChannelNumber, Tdi, Lines */
  Group = Profile
    Name = GainFlatField
    Module = GainFlatField

/* Define the A matrix file reference */
     A = "$mro/calibration/matrices/beta/A_TDI{TDI}_BIN{BIN}_hicalbeta_????.cub"

/* As of version 0020, 9 March 2010, These have the same filename with a .csv extension.*/
/* As of version 0020, 9 March 2010, These coefficients are now called Flats. */
    Flats = "$mro/calibration/matrices/beta/A_TDI{TDI}_BIN{BIN}_hicalbeta_????.csv"
    FlatsColumnName = "{CCD}/{CHANNEL}"
  End_Group

/* This profile contains parameters pertinent to processing */
/* temperature-dependant gain correction.  Formally in GainFlatField module. */
/* Required label keywords: CpmmNumber, ChannelNumber, Samples */
  Group = Profile
    Name = GainTemperature
    Module = GainTemperature

/* Define temperature-dependant gain correction CSV file */
    FpaTemperatureFactorSkipLines = 4
    FpaTemperatureFactorHeader = True
    FpaTemperatureFactorFile = "$mro/calibration/matrices/beta/FpaTemperatureGain_BIN{BIN}.????.csv"

/* As of version 0020, 9 March 2010, This is now called the FPAGain. */
    FPAGain = "$mro/calibration/matrices/beta/Temperature_Gain_????.csv"
    FPAGainColumnName = "{CCD}/{CHANNEL}"
    FPAGainRowName = "{BIN}"

  End_Group

/* This profile contains parameters pertinent to processing I/F conversion. */
/*  Required label keywords: ScanExposureDuration */
  Group = Profile
    Name = GainUnitConversion
    Module = GainUnitConversion

/* I/F correction for tdi/bin - currently set at 1.0 for all tdi/bin */h
/* combinations. */
    GainUnitConversionBinFactor = 1.0
  End_Group

/* Here are the filter profiles.  All keywords that pertain to a filter set */
/* should be specified here.  FilterGainCorrection are I/F corrections in */
/* units of DN/s. */
/* Added CalFact, CCD QE, Temp dependence correction 2010-04-28 */
  Group = Profile
    Name = RED
    FilterGainCorrection = 157510165.0
    IoverFbasetemperature = 18.9
    QEpercentincreaseperC = 0.0005704
    AbsGain_TDI128 = 6.3688
  End_Group

  Group = Profile
    Name = IR
    FilterGainCorrection = 56392816.0
    IoverFbasetemperature = 18.9
    QEpercentincreaseperC = 0.002696
    AbsGain_TDI128 = 6.9809
  End_Group

  Group = Profile
    Name = BG
    FilterGainCorrection = 114682896.0
    IoverFbasetemperature = 18.9
    QEpercentincreaseperC = 0.00002295
    AbsGain_TDI128 = 6.9738
  End_Group

  Group = Profile
    Name = IR10_1
#   LastGoodLine = 3100
  End_Group


  Group = Profile
    Name = Debug

/** Current disables writting to label history due to bug in keyword formatter in ISIS **/
/* The bug has the following error signature: */
/*        terminate called after throwing an instance of 'std::out_of_range'  */
/*          what():  basic_string::substr                                     */
/*         Abort                                                              */
/* You must set this to false when this occurs as a workaround and use the */
/* DumpHistoryFile parameter to see the parameter history. */
    LogParameterHistory = False

/* Uncomment this line to write parameter history to the ProductId log */
   DumpHistoryFile = "{OPATH}/{ProductId}.{Program}.log"

/* Uncomment this line to dump Module data for every module when using Debug */
/* profiling. */
    DumpModuleFile = "{OPATH}/{ProductId}_{Module}.log"
  End_Group

End_Object
            

The filter profiles each contain a scalar constant of the same name for each of the three filter sets. The ProfileOrder keyword contains the {FILTER} pattern that will select the appropriate filter gleened from the label. Other profiles that satisfy the remaining patterns are excluded but can be added when necessary.

The final resutling matrices, constants and keywords used in the calibration equation are recorded in the RadiometricCalibration group of the output label.


Application Control Parameters

Description of main hicalbeta configuration parameters. These parameters govern profile loading, filename pattern replacement and debugging operations. Note that keywords in this section are present in all subsequent profile/module loading.
Parameter Description
Program This keyword is set to the name of the application and can be used to create unique filenames as described previously.
Name The name of the current profile. This keyword is required and is present in the Object keyword section as well as in all other Profile groups. This uniquely identifies the final comingled profile.
DefaultProfile This names the default profile that is loaded when none are specifically called for in the application. It can be any profile but is generally the Object profile (Debug might be an interesting alternative). This will cause no other profiles to be added when a generic one is retrieved.
PropagateTables Specifies the behavior of the table propagation of the FROM file to the TO file when the file is completed. This has some interesting implications. When False, all Table objects in the FROM file are removed in the TO file, which in effect prevents hicalbeta from being able to run again. However, because of debugging capabilities, one may want to select which calibration modules are run. Setting this to True will propagate all Table objects (BLOBs) in the FROM file to the TO file so that hicalbeta can be run again to apply other modules.
LabelGroups This keyword identifies which Groups in the FROM ISIS cube label are included in profiles. This allows any label keyword in the ISIS label to become available for profiling and filename generation. Its handy for creating very specific profiles for problem images and other debugging purposes.
OptionKeywords This keyword selects other keywords in the configuration file or the FROM label groups as defined in the LabelGroups keyword. This list is used to specify additional profiles that can be loaded (via ProfileOptions) and variable substitution within file names (such as B, G and A matrix files).
ProfileOptions This keyword contains the load hierachy pattern of modules/profiles that are evaluated each time a module is invoked. All values in this keyword are an additional potential module that will be loaded into the existing module keyword definition set if it exists in the config file (provided by CONF). The values enclosed in {} are presumed to be an existing keyword in the current state of the module keyword set. If the keyword exists, its value is used to search for an additional module that will be loaded into the current module keyword list. If it doesn't exist, it is ignored.
FpaReferenceTemperature This specifies the reference temperature, in units of Celsius, used to normalize temperature-dependant radiometric calibration variables. It is specified here since it is used in several modules.
Debug::SkipModule This special keyword exists to provide the ability to complete bypass processing of a module. If this configuration keyword resolves to True in any module configuration profile, then the module is not invoked and the resulting contribution is set appropriately such that it does not contribute to the calibration process. This is very useful for debugging and seeing how each module contributes. Be sure to set ProgagateTables to True if you intend to perform subsequent runs of hicalbeta.
OPATH This keyword is the value of the OPATH parameter entered when the program is executed. It can be used to specify a path where log files are written when that option is invoked. If the user did not specify a value for this keyword, the current path is supplied by default.
DumpModuleFile This special keyword exists to provide the ability to dump data from each Module. Each Module implements a data dump that will be written to this file. The file name can be made up of a combination of any label or configuration keyword. The example provided in the config file section uses the ProductId and the name of the Module with a .log extension. This keyword can be included in individual Module Groups or at the top level which will effectively dump all Modules. Note that the GainUnitConversion Module is excluded from this feature as its data are contained entirely in the history report.
DumpHistoryFile This special keyword exists to provide the ability to dump the Module history from each Module. Each Module maintains a processing history that will be written to this file. The file name can be made up of a combination of any label or configuration keyword. The example provided in the config file section uses the ProductId and the name of the Program with a .log extension.

The following tables describe hicalbeta module processing overview. A general description of the module processing steps and configuration file parameters for the modules are provided. Note that each module contains a Name and Module parameter that are always the module name (e.g., ZeroBufferSmooth). They are excluded from each module description below due to the redundant nature of the keywords and for brevity unless they are set to something other than the module name.

Note that all module group descriptions are profiles. Not all profiles are modules. Multiple profiles can be loaded to fully define a set of module parameters. The order of potential profiles that are loaded in addition to the one requested directly in hicalbeta application is specified by the ProfileOptions keyword as described above.


ZeroBufferSmooth Module Parameters

Reads and processes the BufferPixels from the "HiRISE Ancillary" image BLOB. The data is smoothed and missing data is filled with a cubic spline.
Parameter Description
ZeroBufferSmoothFirstSample Specifies the first (0-based indexed) BufferPixels sample to start the average.
ZeroBufferSmoothLastSample Specifies the last (0-based indexed) BufferPixels sample to end the average.
ZeroBufferSmoothFilterWidth Specifies the width of the filter to smooth and fill gaps in the resulting BufferPixels averages. [Default: 201]
ZeroBufferSmoothFIlterIterations Specifies the number of sequential filters to apply to the averaged buffer pixels before finally filling with a spline. [Default: 2]

ZeroBufferFit Module Parameters

This module uses the ZeroBufferSmooth results and computes a non-linear equation for removal of the drift component. It uses a Levenberg-Marquardt algorithm to solve the equation f(x) = a0 + a1 * linetime + a2 * exp(a3 * linetime).
Parameter Description
ZeroBufferFitSkipFit This parameter can be used to turn on/off the non-linear fitting process described above. When missing or set to TRUE, the result of the ZeroBufferSmooth module is used. If FALSE, the ZeroBufferFit fitting is applied using the parameters provided. [Default: True, meaning it is currently not used]
ZeroBufferFitOnFailUseLinear This parameter can be used to select the behavior of the data produced by the ZeroBufferFit module when the non-linear fitting process has failed (typically due to non-convergence). If set to TRUE, a linear fit to the latter half of the ZeroBufferSmooth (BufferPixel) data is computed. If FALSE or non-existent, the result of the ZeroBufferSmooth module is used as is.
ZeroBufferFitMinimumLines Minimum number lines required to apply the non-linear fit processing. The actual number of lines used is the total lines less TrimLines/Summing. If there are not enough lines, then the result of the ZeroBufferSmooth module (filtered BufferPixel data) is simply passed through as is. This results in the same behavior when the ZdSkipFit parameter is invoked.
MaximumIterations The maximum number of iterations to allow the equation to converge.
MaximumLog Constrains the last term in the equation such that the exponent will not cause an overflow.
AbsoluteError Specifies the absolute maximum error to determine convergence.
RelativeError Specifies the relative maximum error to determine convergence.
GuessFilterWidth The data used to fit the non-linear equation can be optionally filtered before fitting. This controls the width of the filter [Default: 17].
GuessFilterIterations This parameter controls the number of times the fit buffer is filtered prior to non-linear fitting. Set this to 0 to turn off this processing and use the data as is. Note this is in addition to the filtering done in the ZeroBufferSmooth module. [Default: 1].

ZeroReverse Module Parameters

This module uses the Reverse Clocked region to compute the offset that is then subtracted from the result of the ZeroBufferFit module. Processing is governed by some statistical properties of this region. The standard deviation and special counts are compared to tolerances provided in the ReverseClockStatistics file that will trigger the use of a constant mean value for this correction. The result of these values are subtracted from each sample pixel column.
Parameter Description
ZeroReverseFirstLine Specifies the first line of the Reverse Clocked region in the input FROM file "HiRISE Ancillary Image" data to average for dark current correction.
ZeroReverseLastLine Specifies the last line in the Reverse Clocked region in the input FROM file "HiRISE Ancillary Image" data to average for dark current correction.
ReverseClockStatistics

ReverseClockStatistics

This is the name of a file that contains profiles for individual channel images and summing modes for trigger and mean values that govern how the offset component is determined. The default file name is "ReverseClockStatistics.XXXX.conf" where "XXXX" is a version number. Note that the contents of the composed hicalbeta configuration profile is used to provide initial values for the profile loaded out of this file.
Parameter Description
Name This names each image profile. The name is a combination of FILTER(e.g., RED, BG, IR), CCD (1-13) followed by an underscore and the channel number (0, 1) followed by another underscore and the summing mode (0-8). Example: RED0_1_1.
RevMeanTrigger Maximum mean value of the Reverse Clocked data region. Means that exceed this value will instead use this value as the offset correction value for all samples.
RevStdDevTrigger Maximum standard deviation value of the Reverse Clocked data region. Standard deviations that exceed this value will instead cause the RevMeanTrigger value to be used as the offset constant for all samples.
RevLisTolerance The maximum number of low instrument saturation (LIS) special pixels that can exist in the Reverse Clocked calibration data region. If more LIS pixels exist in this region, then the RevMeanTrigger value is used in lieu of the processed Reverse Clocked data. This value provides the default (1) for all HiRISE images which can be changed/overridden in subsequently loaded profiles (such as the one loaded in the ReverseClockStatistics file).
RevHisTolerance The maximum number of high instrument saturation (HIS) special pixels that can exist in the Reverse Clocked calibration data region. If more HIS pixels exist in this region, then the RevMeanTrigger value is used in lieu of the processed Reverse Clocked data. This value provides the default (1) for all HiRISE images which can be changed/overridden in subsequently loaded profiles (such as the one loaded in the ReverseClockStatistics file).
RevNulTolerance The maximum number of null (NUL) special pixels that can exist in the Reverse Clocked calibration data region. If more NUL pixels exist in this region, then the RevMeanTrigger value is used in lieu of the processed Reverse Clocked data. This value provides the default (1) for all HiRISE images which can be changed/overridden in subsequently loaded profiles (such as the one loaded in the ReverseClockStatistics file).

ZeroDark Module Parameters

The dark current subtraction consists of a correction based on the "B-matrices" and a temperature correction to these B-matrices, based on a reference FPA temperature and the average of the two FPA temperature measurements (FPA_POSITIVE_Y_TEMPERATURE and FPA_NEGATIVE_Y_TEMPERATURE) taken just before image acquisition. The B-matrices are generated from the residuals left from the previous steps performed on (some) dark frame (night-side) calibration imaging. The temperature correction is a 256-column linear fit (y=mx+b, where m and b are each 256 columns) to correct for the FPA temperature of the image based on a reference temperature of 21 degrees C (this is the FpaReferenceTemperature parameter described above). The 256 columns were used because of the low number of in-flight dark frame images acquired--all bin modes were utilized to generate these linear fits. hicalbeta does a rebin to expand or contract the bin4 linear coefficients to bin8, bin2, or bin1, depending on the bin mode of the image being calibrated.
Parameter Description
DarkCurrent This is the "B-matrix" as generated from the residuals left from the previous calibration steps performed on (some) dark frame (night-side) calibration imaging.
DarkSlope Specifies the file pattern containing the slope component (m) of the Temperature profile in the above "y=mx+b" equation.
DarkIntercept Specifies the file pattern containing the intercept (b) component of the Temperature profile in the above "y=mx+b" equation.
ZeroDarkFilterWidth Width of the smooth filter component for the temperature profile applied after the data has been expanded or contracted to fit the binning mode of the image.
ZeroDarkFilterIterations Specifies the number of sequential filters of width ZeroDarkFilterWidth to apply to the resulting temperature profile after expanding or contracting of the data. [Default: 1]

GainLineDrift Module Parameters

The GainLineDrift module in hicalbeta uses the line correction coefficients which are contained in the file as patterned by the LineGainDrift as defined below. This file contains 4 parameters (C[1234]) and is applied using the equation ""C1 + C2 * LT + C3 * e xp (C4 * LT)" where LT is line time in seconds based upon binning and line exposure duration (from the ScanExposureDuration label keyword).
Parameter Description
LineGainDrift Specifies the file pattern containing the 4 line correction parameters (C[1234]) of the gain/line equation as described above.
LineGainDriftColumnHeader This parameter specifies whether the LineGainDrift file contains a header line to skip. It is a True/False boolean value. It is needed here to notify the generic CSV parser a column header exists but is not used to resolve the extraction of values from the file.
LineGainDriftRowName Specifies the desired row header name contained in the LineGainDrift file from which to extract the applicable 4 parameters. This must be specified to select a specific given row. Its presence in the config file implies a row header exists in the first column of the LineGainDrift file. The value of this parameter is typically specified in the config files as a pattern combination of the CCD and Channel of the image being calibrated.

GainNonLinearity Module Parameters

The GainNonLinearity module in hicalbeta uses the nonlinearity coefficients that are derived as a function of binning mode. These coefficients are used in the following equations to correct for nonlinear gain (signal-dependent gain) "GNL = 1 - (GNLc * <Line>)" where GNLc is the GainNonLinearity coefficient (CCD and Channel dependent) derived as described below, and <Line> is the average of the line of the image being processed.
Parameter Description
NonLinearityGain This is the file pattern containing the GNLc parameter in the above equation.
NonLinearityGainRowName Specifies the name of the row in the NonLinearityGain file to extract the coefficient from. The existence of this parameter implies the first column in this file contains a named header for each row. The value of this parameter is typically specifed in the config files as a pattern combination of the CCD and Channel of the image being calibrated.

GainChannelNormalize Module Parameters

The GainChannelNormalize module in hicalbeta uses the bin, TDI, CCD, and Channel dependent coefficients described below in the equation "GCN = GCNc * 128 / (TDI * bin^2)" where GCNc is the normalization coefficient extracted from the Gains file below, and the final term is the normalization to bin=1, TDI=128.
Parameter Description
Gains This parameter is the file pattern of the "G-matrix" containing the GCNc parameter in the equation above.
GainsRowName This parameter specifies the name of the row in the Gains file from which to extract the GCNc parameter. Its existence implies a row header exists in the first column of the file. Its value is a function of the binning mode of the image being calibrated.
GainsColumnName This parameter specifies the name of the column in the Gains file from which to extract the GCNc parameter. Its existence implies a column header exists in the first row of the file. Its value is a combination of the CCD and Channel of the image being calibrated.

GainFlatField Module Parameters

The application of the "A-matrix" flatfield correction.
Parameter Description
Flats This parameter is the file pattern of the "A-matrix" containing the per-sample flatfield correction parameter.
FlatsColumnName This parameter specifies the name of the column in the Flats file from which to extract the flat field parameter. Its existence implies a column header exists in the first row of the file. Its value is a combination of the CCD and Channel of the image being calibrated.

GainTemperature Module Parameters

This module contains parameters specific to temperature dependant gain correction.
Parameter Description
FPAGain This parameter is the file pattern of the FPA factor for the temperature dependent gain coefficient.
FPAGainColumnName This parameter specifies the column name pattern of the desired parameter in the FPAGain . The value of this parameter is typically specifed in the config file as a pattern combination of the CCD and Channel of the image being calibrated.
FPAGainRowName Specifies the desired row header name contained in the FPAGain file from which to extract the applicable parameter. This must be specified to select a specific given row. Its presence in the config file implies a row header exists in the first column of the FPAGain file. The value of this parameter is typically specifed in the config file as a pattern for the binning/summing node of the image being calibrated.

GainUnitConversion Module Parameters

This module computes values necessary to convert DNs to user selected calibration unit values.
Parameter Description
GainUnitConversionBinFactor Specifies the factor that takes into account binning of the image data. It is currently set to 1.0.
FilterGainCorrection Specifies the filter dependent gain correction factor. Note that these values are found in the BG, RED, and IR profiles and hicalbeta relies on the profile options to load the proper parameter at runtime.
IoverFbasetemperature Base temperature I/F conversion factor that is filter-dependent. This parameter is found in the filter dependent profiles BG, RED and IR and relies on the profile options to load the proper parameter in at runtime.
QEpercentincreaseperC Measurement of quantum efficiency increase per 1C assuming linear with temperature. This parameter is found in the filter dependent profiles BG, RED and IR and relies on the profile options to load the proper parameter in at runtime.
AbsGain_TDI128 Normalization of absolute gain to TDI 128. All other TDIs are scaled by this parameter. This parameter is found in the filter dependent profiles BG, RED and IR and relies on the profile options to load the proper parameter in at runtime.

Categories


History

Drew Davidson2005-06-27 Original version
Drew Davidson2005-06-27 Wrote application test
Drew Davidson2005-08-17 Added examples
Kris Becker2005-10-03 Added processing of DarkPixel object data. The median of the 16 dark pixels in subtracted from the original DN. Also added the DARKSTATS output file.
Kris Becker2007-07-02 Extensively modified to implement latest state of HiRISE calibration.
Kris Becker2008-01-12 This version is created from hical as a starting point. It has been again extensively modified for maximum flexibility and usefullness.
Kris Becker2008-02-08 Corrected the DriftCorrect class to properly compute the initial guess of the fit parameters. It was not quite the same as the model. Also added capability to dump individual module data via the configuration file.
Kris Becker2008-03-11 Added the OPATH parameter and made it available for subsititution in the configuration parameters; Added the ZdSkipFit configuration parameter to skip the non-linear fitting process in the Zd module (when invoked, it will use the result of the Zf module); Added in the percent additional contribution for the FPA temperature/gain correction; Added ZdMinimumLines option to constrain the size of an image that is fitted (properly).
Kris Becker2008-04-01 Added missing factor (128/tdi/bin^2) to Zgg G-matrix gain module. Explicitly recognize Phobos and Deimos targets as Mars so that I/F can be computed.
Kris Becker2008-04-02 Fixed removal of HiRISE calibration ancillary BLOBS (tables) so that it only removes these tables and not all others, such as SPICE tables.
Steven Lambright2008-05-12 Removed references to CubeInfo
Kris Becker2008-05-23 Added the ZdOnFailUseLinear flag to specify behavior when the non-linear DriftCorrect Zd module fails.
Kris Becker2008-06-11 Change default Zf filter defaults to 201 width with 2 interations; Add reporting of raw buffer data to Zf module data dump; Added computation of average and standard deviation to difference of raw buffer and post filtering in Zf module; Set Zd default behavior to skip LM fitting and eliminate unnecessary linear fitting in skip mode.
Kris Becker2008-06-13 Abstracted Buffer pixel processing to DriftBuffer class (Zf); Added robustness to some module dumps (Zf, Zb, Za); Added statistics computations to several modules (Zf, Zz, Zb,Za); Abstracted dark offset processing of Reverse Clocked data to OffsetCorrect class (Zz).
Jeannie Walldren2008-11-05 Replaced references to DataInterp class with NumericalApproximation.
Kris Becker2008-11-17 Added additional code to implement Zz module trigger options. This option provides the ability to trigger a constant used in the Reverse Clocked calibration data based upon noise. See the discussion on the Zz module in this program documentation.
Kris Becker2009-09-15 Change default for OPATH to output (TO) directory; Removed the IOF parameter and replaced it with the UNITS parameter; Added the Zt module which corrects temperature dependant gain previously in the Za module; updated documentation.
Kris Becker2010-04-27 Added generalized CSV file access; renamed many of the modules names to be better maintainable; moved all module implementations to classes/headers with the same name.
Kris Becker2010-05-20 Added GainNonLinearity module and support for it. It now requires two loops through each line as a median value in mid-calibration is needed in subsequent application of this new module's contribution. All ancillary file access is using the new generalized CSV format.
Kris Becker2010-05-26 Corrected sign for temperature gain coefficient.
Kris Becker2010-11-04 Renamed many of the modules to better reflect their function. Extensively updated the program documentation.

Parameter Groups

Files

Name Description
FROMInput cube to calibrate
CONF File containing HiRISE calibration and matrix configuration parameters
TO Output radiometrically corrected cube file

Options

Name Description
PROFILE Name of specific configuration profile to use
UNITSDesired output calibration units
OPATH Alternative path to use when logging is invoked
X

Files: FROM


Description

The name of the cube to which the correction will be applied. The correction will apply to every non-special pixel in the image.

Type cube
File Mode input
Filter *.cub
Close Window
X

Files: CONF


Description

This file is critical to the proper functioning of hicalbeta. It contains parameters that are loaded to provide necessary data used in the calibration of HiRISE CCD channels. It is extensively documented in the main documentation section of this application.

Type filename
File Mode input
Default Path $mro/calibration
Default $mro/calibration/hicalbeta.????.conf
Filter *.conf
Close Window
X

Files: TO


Description

Use this parameter to specify the name of the output cube. If you do not include an extension of ".cub" it will be added automatically.

Type cube
File Mode output
Pixel Type real
Close Window
X

Options: PROFILE


Description

This is the name of a named Profile group in the hical matrix configuration file that users can choose for specific selection of HiRISE calibration parameters and matrix files.

Note that when using this option, the additional profiles specified in the ProfileOptions key are not applied. This option is primarily to assist in testing and specific application conditions.

Type string
Internal Default None
Close Window
X

Options: UNITS


Description

A feature which calculates the average on each channel and multiplies one side with the ratio of the averages

Type string
Default IOF
Option List:
Option Brief Description
IOFIrradiance over flux Commonly referred to as "I over F", this option selects the output calibration units as irradiance over flux.
DN/USData number per microsecond This option selects the output calibration units to be data number per microsecond.
DNData number This option selects the output calibration units to be data number.
Close Window
X

Options: OPATH


Description

This parameter is used to specify a directory path that is intended to be used in output file specifications. OPATH can be useful for directing specifically where log files are written when that option is invoked. This option is automatically added and available to all profiles and is included in the output specifications of all log files.

The default behavior is to place all log files in the same directory as the output file specifed in the TO parameter.

Type string
Internal Default None
Close Window