INTEGRATE 2.3.3

Program Notes: 31 July 2000

General 

This program searches a group of concatenated merge-type files to locate sets of contiguous sections through crystals. It then analyzes these sections to determine whether they are: (1) small fragments of crystals at the upper or lower limits of the scanned volume, which should be excluded from the output data set; (2) larger incomplete crystals near the upper or lower limits of the scanned volume, whose centers and radii can be identified accurately enough to permit their inclusion in the output data set; (3) multiple crystals in contact with one another, which must be separated into two or more individual crystals; or (4) "normal" crystals, which are whole individuals. Once a set of contiguous sections is processed in this way (either automatically by the program for uncomplicated cases, or with operator intervention for ambiguous cases), then the sectional volumes are integrated to define a spherical crystal with equivalent volume whose center is located near the centroid of the contiguous sectional cylinders.

As currently written, Integrate is meant to be used in tandem with Blob, identifying problems with the data in Blob, making corrections in Blob, and merging the corrected slice data into Integrate.

Memory Required: depends on input size

In processing George, a volume with 86 slices and about 3000 crystals, the maximum memory required was about 2500 kb.

Input File Type

A composite of sequential merge files that have been concatenated in a word-processing program (run together end-to-end with header information included) or by the ConcatText utility program, or a PRESERVE file created by this program.

Input Limits

Inputs are now limited only by available RAM, although there may be bugs related to variable types used to identify sections and slices that might appear if either exceeds 32767.

Output File Types

integrate (old-text), Integrate (new-Binary),

PRESERVE.

ResTypes (not registered with Apple)

Application (Integrate) = ‘qtIN’

PRESERVE file = ‘iNsV’

INTEGRATE file = ‘iNgT’

Operator Notes

  1. The 'Display' menu selection toggles back and forth between the options 'Show Normal Crystals' and 'Skip Normal Crystals'; the active choice will have a checkmark beside it. If 'Show...' is chosen, then every set of contiguous sections will be displayed for operator intervention. If 'Skip...' is chosen, then the program will automatically process all crystals that it can recognize as 'normal' and will request operator intervention only for the ambiguous cases. Criteria for determining "normalcy" are discussed below in the program notes.
  2. The consequences of the four possible decisions offered to the operator (via check boxes) when he/she is asked to intervene are listed below. They can be selected both with the mouse (as expected) or from the keyboard by just hitting the first letter of the desired option; i.e. ‘n’ or ‘N’ for ‘Normal’, etc. If the active window is not the decision window, then the decision window will become active upon clicking (of course) and also upon typing one of the four response letters.
    1. Normal: In this case, the x- and y-coordinates of the center of the crystal are taken as equal to the means of the set of x- and y-coordinates of the individual sections of the crystals, and the z-coordinate is at the level that separates 50% of the volume (interpolated from the slice below which is less than half of the volume to the slice below which is more than half of the volume. The radius is that of a sphere with a volume equal to the total volume of the contiguous sections.
    2. Partial: In this case, the x- and y-coordinates of the center of the crystal are set equal to the means of the set of x- and y-coordinates of the individual sections of the crystals, and the z-coordinate is placed in the center of the slice designated by the operator in the decision dialog. The crystal radius is taken as the radius of the section in the slice designated by the operator in the decision dialog.
    3. Multiple: In this case, all slices below and including the "topmost" slice designated by the operator in the decision dialog are grouped together and evaluated by the algorithm as a set of sections. If necessary, these sections are presented to the operator again for a decision. All higher slices are returned to the data file to be read again later. For example, if there is a set of slices that comprises three crystals, this procedure will allow the lowest crystal to be removed first, and the remaining two crystals would appear again later as a pair, which would have to be subdivided again.
    4. Exclude: In this case, all slices in the set are simply ignored.
  3. In addition to the above option, the buttons on the right-hand side of the "decision" dialog offer one other actions. The Redo last one button takes back the last action the user selected. It can only "go back" one action.
  4. Show Stack in Blob is used to highlight all the circles that make up the current stack in the program Blob. Only currently-open windows that make up the stack will be affected.
  5. Replace slices allows revised slice data to be incorporated into the current data set. A number of merge files with revised data is selected, and these are read in. Integrate then "starts over" from the bottom of the data set, and any stacks that have changed must be re-evaluated by the algorithm and/or the user.

Program Notes

  1. To identify a set of contiguous sections, the program begins with a section in the bottommost slice, then searches the next slice above for the section having its center closest to the x- and y-coordinates of the center below. Having found that closest match, originally (through version 1.0.4) it checked to see if either the section above overlapped the center point of the section below, or if the section below overlapped the center point of the section above. This was changed in version 1.0.5 and subsequent versions to check if any part of either crystal overlaps any part of the other. If overlap is found, then the higher section is added to the contiguous set, and the next slice up is searched in the same way. Version 1.0.10 added the further criterion that once an overlapping crystal is found, the slice below it is searched to make sure the section being stacked upon is the closest in that slice to the overlapping circle; thus, if the two sections are not the mutually closest, they are not paired. When no overlap is found, the higher section is regarded as belonging to a different crystal, and the set of accumulated sections is closed and referred to a processing routine described below.
  2. The following criteria are applied in the automatic processing of sets of crystal sections:
    1. If the sets of sections begin or end in the topmost or bottommost slices, and if there are only one or two sections, then those sections are automatically excluded.
    2. If the sets of sections begin or end in the topmost or bottommost slices, and if the vertical thickness of the set of sections is less than 60% of the diameter of the largest section in the set, then those sections are automatically excluded.
    3. If the sets of sections begin or end in the topmost or bottommost slices, and if the vertical thickness of the set of sections is less than the diameter of the largest section in the set (but greater than 60% of it), then those sections are referred to the operator with the expectation that they are a partial set.
    4. If there are at least two sections in the set, and if the vertical thickness of the set of sections is less than 50% of the diameter of the largest section in the set, then those sections are referred to the operator with the expectation that they are a normal set.
    5. If there is only one section in the set, and if the vertical thickness of the set of sections is less than 25% of the diameter of the largest section in the set, then those sections are referred to the operator with the expectation that they are a normal set.
    6. If the vertical thickness of the set of sections is more than 150% of the diameter of the largest section in the set, then those sections are referred to the operator with the expectation that they are a multiple set.
    7. The sectional radius of a normal crystal should start off small, increase and then decrease again. If the set of sections contains at least 3 sections, the program checks for this by smoothing the radius vs. slice number data gently [R(i):= 0.25*R(i-1) + 0.50*R(i) + 0.25 R(i+1)] and then examining the slopes on a plot of radius vs. slice number. To be considered automatically as a normal crystal, that plot must start off with positive slope, and must reverse its slope only once in that set of sections. A set of 3 or more sections that don't plot up in this way is referred to the operator with the expectation that it is a multiple set. [A set of one or two sections that is not excluded or referred to the operator on the basis of criteria (a)–(f) above is automatically considered to represent a normal crystal.]
  3. In order to handle the replace slices functionality, the program keeps a list of all decisions made and the stack that the decision was for. After replacing slices, it removes the current list of found crystals, and starts over at the beginning. Whenever it finds a stack for which there needs to be a user decision, though, it first checks the decision archive to see if that exact stack had a decision made before. If so, it uses the old decision, and moves on, without user interaction. All valid decisions are placed into a new decision stack, and the old one is tossed out.

Known Bugs

  1. At upredictable times, the image in the display window will not appear centered. By sizing the window smaller, and then larger, it is possible to work around this bug at run-time.
  2. The "processing decision archive" status sometimes requires one to finish the current slice before it is dismissed and saving is enabled.

Possible Bugs

As noted above, there may be bugs related to variable types used to identify sections and slices that might appear if either exceeds 32767.

Release History

1.0.0 Initial release.

1.0.1 Added to the output both the slice number and the timestamp of the central section through the crystal.

1.0.2 11 June 91

1.0.3 14 June 91

1.0.4 16 June 91

1.0.5 mid summer 91 ?

1.0.6 18 Dec 91

1.0.7 19 Aug 92

1.0.8 4 Jan 94

1.0.9 13 Feb 94

1.0.9a 15 Feb 94

1.0.10 2 Mar 94

2.0.0 7 July 1995

2.0.1 17 January 1996

2.1 14 February 1996

- PRESERVE file is binary, and only saves the necessary data (reduces PRESERVE file size on Claude from 17MB to 900KB!)

- custom icons for application, PRESERVE files, and export (INTEGRATE) files

- can switch from skip normals to show normals at any time

- made it a fat application: will run on both Power Macs and 68K macs.

2.2 14 April 1999

2.3 24 October 1999

2.3.1 11 November 1999

(Program Notes hiatus)

2.3.3 31 July 2000

Bug fixes.

Authorship

The first version of this program was written by William Carlson in late May 1991. Versions from 1.0.7 to 2.0.0 done by Richard Ketcham, dates as listed. Versions from 2.0.1 forward done by Dave Hirsch, dates as listed.