Everything Else
Compacting and Rescaling Databases in Helix Utility 7.0
Introduction: Two Problems

Helix databases function as a self-contained collection of ‘objects’ making backup and transport of a Helix database as simple as copying a single icon. Internally, these objects are tracked in a ‘map’ that can, over time, become filled with gaps as old structure is deleted, whether by the collection designer or as the daily use of Helix causes temporary objects (e.g. queries) to come and go. A large object map with many gaps can cause performance degradation due to the increased workload of searching for open areas to store new object information.

Helix users have been able to accumulate over 30 years worth of data in a collection. For some businesses, this means that the 4GB limit of a Helix collection presents a problem, requiring them to ‘archive’ (that is: delete) old data to stay under that threshold.

Problem One Solved: Compress

When compressing a collection, Helix Utility 7.0 offers two new checkbox options, as seen in the image on the right:

  1. Remove Free Space: This is the function that compressing a collection has always performed. It moves all of the objects together, eliminating gaps in the collection file itself, thereby reducing the overall size of the collection. Although modest performance improvements might be seen after compressing a collection, there was usually little benefit to doing so unless significant amounts of data had been deleted from the collection.
  2. Compact Object Map: This is the new function that provides further size reduction and potentially significant performance improvements by rewriting the master map that tracks where each object is found to remove empty spaces. The immediate effect of a compacted object map is faster saves; the other main effect is faster performance overall.

The image on the right shows the ‘Compress’ panel with some information pre-loaded. The first ‘Compression Info…’ section shown in the Activity Log (acquired by clicking the ‘Get Info’ button on the Compress panel) writes five lines of information to the Activity Log:

  1. The internal collection format: useful mainly when diagnosing problems
  2. The current size of the collection: 4140MB (4.14GB) in this example
  3. The estimated potential size reduction after compression: no savings in this example
  4. The estimated size of the structure of the collection: mostly informational
  5. How many entries are in the Object Map, and how many are currently in use

In this example, removing free space will have little to no effect — the collection has few gaps between actual data. However, 45% of the Object Map entries are unused, meaning that some benefit would be gained by compacting it. The collection would not be any smaller after compression, but it would be faster.

Note: Helix Utility can be used with older collections, and the new ‘Compress’ options are fully compatible with all Helix versions.

Problem Two Solved: Rescale

Helix Utility 7.0 contains a new ‘Rescale’ panel that can be used to allow a collection can now grow to as large as 64GB, an eightfold increase in storage capacity. Understanding how this is accomplished is important for maintaining optimal performance settings.

The collection size is limited by the total number of ‘blocks’ of data the current collection format allows. By adjusting the ‘internal block size’ — the base storage unit of a Helix object — of these blocks, a collection can expand to larger sizes. Since most objects span multiple blocks, enlarging the block size means any given object will require fewer blocks, thereby forestalling the possibility of reaching the block count ceiling.

Using fewer blocks to store an object also adds a degree of efficiency, and object will be less likely to be ‘fragmented’ across more than one block. (Those of you who understand these concepts as they apply to hard drive technology will immediately comprehend.)

However, not all internal objects require more than a single block. In those cases, an expanded block size becomes less efficient, as a collection storing the exact same amount of data will require more disk space in which to store it. By allowing expansion in gradients, collections that do not require extreme size can run more compactly.

The image on the right shows the ‘Rescale’ panel with some information pre-loaded. The ‘Compression Info’ section shown in the Activity Log (acquired by clicking the ‘Get Info’ button on the Compress panel) shows that this collection is currently 4,140 MB, or just over 4GB. Clicking the ‘Get Info’ button on the Rescale panel produces different information, as seen in the ‘Rescaling Info’ section shown in the Activity Log.

The five lines in the ‘Rescaling Info’ section tell you (approximately) what will happen if your collection is rescaled to each possible size. Each line details:

  1. Which rescale option this line is describing
  2. Whether this is the current scale or a rescale option
  3. The resulting collection size, given the current structure and data in the collection
  4. How much room for growth that size affords
  5. How ‘full’ the current collection would be (or is) at that size

Each line is also color-coded to indicate minimal headroom situations. Sizes that would result in a collection more than 90% full are color-coded in Orange; more than 95% full in Red.

The contextual menu that appears when clicking on a collection name now includes a ‘Get Rescaling Info’ item to trigger the ‘Get Info’ button in this panel; the old ‘Get Info’ command has been renamed ‘Get Compression Info’ to clarify which info that shows.

Note: Although Helix Utility can be used with older collections, the ‘Rescale’ option can only be used with collections updated to Helix 7.0 or later. A warning appears when attempting to rescale an older collection, and the operation does not go forward.

Expert Panel

The ‘Expert’ panel is only for customers who have completed collection repair training. There are currently no collection repair training sessions scheduled, but they may be offered again, if demand warrants.