Helix RADE 6.2 Preliminary Release 5: AppleScript Nirvana, and other news

Get Your Upgrade Here

10 October 2011 —The good progress continues with the fifth Preliminary Release of RADE available today.

For those of you who have already climbed aboard the Europa Pioneer plan, your monthly $20* charge will appear on your credit card today. For those out there among “the rest of you,” remember that if you’re thinking of joining the Pioneer program this month, your initial payment is $100. After all, you are getting “$100 worth of improvements” right from the start. All subsequent monthly payments will be $20.

This month’s release continues to enhance RADE’s AppleScript capabilities, adding more key pieces of the puzzle. The finish line for this phase is clearly in sight!

Sadly, we’ve had a few people drop out of the Europa Pioneer Plan. This created a small conundrum for us, as we had made the update process a simple matter of opening the About Helix RADE window (in the Helix RADE menu) and clicking on the download link. The new process is almost as easy: when your credit card is successfully charged, we send you an email. In that email will be a link to the newest installer. Click that link to download the lastest RADE to your computer.

Once you’ve got the installer, the process remains the same: If this is your first experience with Europa — or you like to manage your updates manually — just drag the new Helix RADE onto the Applications folder and let go. If you are updating from an earlier version of Europa, double click the new Install Update icon on the disk image. Your Europa RADE will be seamlessly updated to the new version, even if you have a collection open! (You will be prompted to save changes before closing an open collection.)

What can the new Helix RADE — Europa — do today?

For starters, it runs. In fact, it runs so well you can… well, let us not get ahead of ourselves.

It self-diagnoses. If you are a Europa Pioneer and have been running Helix RADE 6.2 builds on an Intel-based Macintosh, you have already experienced the benefit of having Helix automatically run the structure check for you, performing routine maintenance as necessary.

Now we know, of course, that some of you are completely nonplussed by this, but if you don’t appreciate it, just consider that in the long run, routinely maintained Helix collections will have fewer problems and will make for happier customers. That’s you. The days of working for hours only to discover that a bit of structural damage has crept in during the process are over.

Now, if you attempted to use the first Europa releases on a PowerPC Mac, you probably did not have much luck doing anything with all but your smallest collections, because the integrated structure check appeared to hang up, when in reality it was running… very slowly. This was due in part to the fact that the integrated structure checker was cleaning up anomalies and obsolete structure that had previously gone undetected. We’ve literally seen hundreds of thousands of these little bits of uselessness in some old collections!

In order to keep the PowerPC users going, our last release gave you a way to turn the integrated structure check off, so you could work — and save your work — at last. But when you turn the integrated structure checker off, you have to remember to run your collections through the old Update Collection 6.0 application to check the structure.

If you are serious about Helix and you work with large collections, you probably already have an Intel Mac. If you haven’t already done so, open your production collections with the latest Europa on an Intel Mac and clean that stuff up. If you don’t have an Intel Mac yet, maybe it’s time. But we digress.

With the ability for PowerPC Mac users to bypass the integrated structure check, they have been able to do virtually everything else the Intel-based Mac user can. But that may be the last nod we make to the PowerPC Macs. We are no longer intentionally funneling any resources into making things work better on PowerPC Macs, which were last manufactured in late 2005. But we are being careful to continue to write Universal code that runs on both Intel and PowerPC Macs, mainly because there is no extra effort involved.

We are soon going to conclude the AppleScript implementation phase of Europa, and we will then turn our attention back to the GUI. We have some ideas about how to do that most efficiently, but some of them may require extra effort for the PowerPC. If that becomes evident, those functions will simply be disabled on PowerPC Macs. The future (and present, really) is on Intel.

Just a few short months ago, using Preliminary Release 1 of Helix RADE 6.2, you could open and run a copy of your Helix collection in User Mode, just as you could do in Helix Engine, and for the first time in macOS, you could go into Design Mode and have a look around. But that was about all you could do. And back then, even though the User Mode part of RADE was ostensibly the same as what you see in Helix Engine 6.1, it was not yet as stable.

What can you do with Europa?

Up above, we wrote that Europa “runs so well that you can…” and then we left you hanging. Let’s finish that sentence: Europa runs so well that you can now, starting with Preliminary Release 5, officially work in User Mode for daily tasks. If you are using Helix Engine to use your Helix collections in User Mode in macOS, feel free to switch over to Helix RADE 6.2b5 without fear of damage. Should something completely unexpected happen and you do encounter damage, we’ll do the repair for free, unless the evidence points to something else (like a failing Hitachi hard drive) as the culprit.

None of the foregoing, however, should be taken to imply that you are safe to run without making frequent backups of your work and checking both the structural and data integrity of your collections!

With most of the icons now scriptable, you can actually pop in and out of Design Mode and accomplish things. All of the icon types can be created in the GUI, and you can use the macOS Inspector to edit the attributes of fields. The GUI also lets you quickly and easily delete the data from one field or a whole relation, and the new Used By window — the updated replacement of “Get Info” must be seen to be believed. When you add in the AppleScript capabilities, you can now create and edit all of these Helix collection elements in macOS RADE:

1. Collections
2. Relations
3. Fields
4. Indices
5. Form Queries
6. Power Queries
7. Templates, for entry or listing
8. Views, and specify the template, query, indexes, posts, and other attributes
9. Users, setting command keys, permissions, and set the window positions (something you can’t do in Classic!)

You can also perform global search and change operations, along with a vast array of functions not previously possible in Helix RADE.

You can use AppleScript to manipulate and deploy existing icons of all types. We’ve got scripts that resize templates, that map (and remap) the colors used in a collection, change a view’s location and permissions for every user, regardless of whether they have it on their menus or not. We’ve got a script that breaks only the keyword indexes, something that could previously only be done by manually locating every keyword field in the collection, opening them one by one and toggling the keyword checkbox off, then back on. You could have also used the Break All Indexes command in Helix Utility, but that rebuilds every index. Another script replicates the Quick Entry and Quick List functions, but is completely customizable so you can tailor the font, size, spacing to your liking.

We could go on and on, but one thing should be clear: if you’re willing to invest the time in learning — like you did when you first learned how to build structure in Helix oh, so many years ago — you can accomplish an awful lot right now.

You may be asking yourself: if they have all of these scripts, why don’t they let us see them? Good question. Three answers: The first one is that we are holding most of them back and making them available exclusively to Learning AppleScript with Helix subscribers along with the tutorials. The second is that in some cases the scripting capabilities of Helix have been growing so rapidly that we haven’t had time to test them thoroughly yet.

The final answer is: we actually are making them available. Well, some of them anyway. If you haven’t seen it, hop on over to our Scripts page and see what we’ve made available. We’ll be adding many more scripts to this page as time goes by.

So what can’t you do in macOS RADE yet? You cannot edit sequences, posts, or abaci. You can create them and (re)name them, but that’s not really much to brag about. To edit these three icon types, we must still rely on Classic RADE. But not for much longer, and we’ll talk about that a bit more later. When that day comes of course, dependence on Classic finally ends.

Two steps back, one step sideways, and a big leap forward

With a simple click of the mouse, you can find that we went out on the limb a bit last month with the following prediction: “On October 10th we expect to deliver a Helix RADE 6.2 with AppleScript support for these icons: the sequence, form query, power query and post.”

October 10th is now here and we are forced to take ‘two steps back,’ delivering only two of the four expected icons; the form query and power query. We are, naturally, disappointed that we made a commitment we couldn’t keep, but read on and we think you’ll forgive us.

Terminology Alert! Helix has — for many years — supported four distinct types of users. Historically they have been referred to as user, submenu, administrator, and prototype. Apart from how almost silly it seemed to call a user a “user user,” the “sub menu” concept has historically been confused by many Helix users with heirarchical menus. So we decided to take ‘one step sideways’ and change the terminology: the user user type is now referred to as a primary user, and the sub menu user type is now referred to as a secondary user. Hopefully these new terms will clarify the role of these two user types and clarify the purpose of the secondary user.

The ‘big leap forward’ is why the sequence and post icons were delayed. Because of some nagging issues that kept cropping up, we decided it was time to finally tackle a project that has been on a back burner since we began this project. It’s something we’ve internally — and only half-jokingly — named AppleScript Nirvana.

What’s new in Preliminary Release 5

As always, we found new bugs to fix, and filled in a few minor missing pieces. The release notes for Preliminary Release 5 provide a detailed look at the tremendous amount of work that has been done since September 10th, but these are the highlights:

Form Query & Power Query Scripting

One of the most powerful features of Helix is giving the user the ability to query, to search through the data that he or she has amassed in any way they like.

The Form Query is often mistakenly referred to as the “Open Query,” which is actually the menu command used to open a query, whether it is a Form Query or a PowerQuery. The two query formats offer significantly different approaches to the data being searched.

The Form Query is the grandfather of the Power Query and is the one most often used by Helix users because a) it was part of Helix from the beginning and b) it uses an existing form template to create an “instant” query. While not the most optimal design method for the form query, that kind of construction is allowed and often provides a simple and intuitive visual metaphor for users learning how to search for information in a Helix database.

What the Form Query also provides is a way for the designer to, well, “design” a query form, making it look exactly as they wish. But the biggest drawback is that the Form Query can only combine your queries using and logic. If you are looking for every customer in Texas, it works great. But if you are looking for every customer in Texas, Arkansas, or Louisiana, the Form Query comes up short.

The Power Query is the much-maligned yet significantly more powerful of the two queries. It is much-maligned because its Classic interface was very difficult for the novice user to understand. Many users were so confused by it that they essentially gave up on it and stuck with the Form Query.

That’s a shame because the main shortcoming of the Form Query is completely gone in the Power Query. With a Power Query, finding every customer in Texas, Arkansas, or Louisiana is easily doable.

Knowing how powerful, yet maligned the Power Query was, we completely rewrote the interface for macOS. It now works very much like the queries you can do in the macOS Finder, in iTunes, and in many other macOS applications. Users old and new who have tried it have embraced it for the intuitive power of its new interface. It is one of the most clearly “designed for macOS” elements of Helix. We like to think is the way this function was always supposed to look.

Regardless of your preference, both of these queries can now be defined via AppleScript. One thing we should note, however: all we have done so far is to make the setup of the queries scriptable. Actually specifying a query is something we strongly considered doing this month, but decided we should draw the line and only implement ‘pure structure’ scripting at this point. Rest assured that when we are done you will also be able to specify a query through scripting as well.

AppleScript Nirvana

Think of AppleScript Nirvana as the “northwest passage” between Helix RADE and AppleScript, a way to dramatically shorten the trip from idea to execution when using AppleScript to direct Helix.

Without Nirvana, using AppleScript to work in Helix can roll right past quirky to annoying. Without Nirvana, you had to ‘navigate the hierarchy’ step by step, using explicit terminology in order to issue a command. Now that’s history. To use a real-world analogy: if you want Fred’s son’s dog to sit, you can now “tell Fred’s son’s dog to sit.” Without Nirvana you could only talk to Fred: you were required to ask him to talk to his son, relaying your request that the son ask his dog to sit.

The net result is the same either way, but with Nirvana the the requirement to ‘nest’ commands largely disappears. Need a real AppleScript example? Here’s how you turn off the ‘keep current’ attribute:

Without Nirvana you had to write this:

tell application "Helix RADE"
  tell collection 1
    tell relation "accounting"
      tell view "Current A/R"
        set keep current to false
      end tell
    end tell
  end tell
end tell

With Nirvana you can now write this:

tell application "Helix RADE"
  set keep current of view "Current A/R" of relation "accounting" of collection "Finance" to false
end tell

Or even this:

tell application "Helix RADE"
  set collection "Finance"'s relation "accounting"'s view "Current A/R"'s keep current to false
end tell

You can also combine ‘Nirvana’ and ‘non-Nirvana’ code at any level — Helix will now ‘get it’ and happily send your commands to the correct place.

AppleScript Nirvana also provides some potential benefits down the road, because it opens the door to Helix being scriptable by other dialects such as Python, Ruby, and other powerful scripting tools that can do wondrous things. At some point we may provide scripting implementations for alternative languages, but if you are a Ruby proponent, you should be aware that there’s a bridge application available now — for free — that make Helix scriptable via Ruby. This long-term advantage alone more than makes up for delaying the work on the sequence and the posting icons.

What comes next?

If we have learned one thing this month, it is the one thing we should probably have learned long ago. Beginning with our predecessor‘s prediction one July day late in the last century that you would see ‘a macOS Native Helix RADE by Christmas,’ we should have learned that prognosticating in print is for fools who don’t know better.

So we won’t say here exactly what November’s edition will bring. But you can rest assured that despite the detours we took this month, we haven’t gotten completely derailed. Once we confirmed that Preliminary Release 5 was ready to go into your hands, our engineers turned their focus back to the sequence and post icons. We certainly hope to see their debut in November, with the final piece of the puzzle, the abacus, coming shortly thereafter. But all we can say with any certainty is that we are working on them in that order.

And for you subscribers to the Learning AppleScript with Helix self-paced training course, the long-awaited “July–August” issue really is coming soon. The problem we are having is essentially that the thing we are trying to teach is going through weekly, and sometimes daily, change. What we have already written often needs revision, and the process of exploring the template in detail (the subject of the next issue) revealed many subtle — and some not-so-subtle — bugs. We thought it was better to delay publication than to give you inaccurate information.

About that ‘A bit more later’ thing

One of the side benefits of working in RADE for macOS is that the solutions to problems that have historically plagued Helix on some level or other are easier to find and fix, working in this modern environment. And while we are, as noted above, not doing anything intentionally to advance the older Helix versions, the last few iterations have indeed benefitted from fixes built for 6.2.

So many Helix problems have been solved over the years that the things that still “plague” Helix are not so much plagues as they are colds and sniffles. But as improvements are made, anything that isn’t right sticks out like a sore thumb. Especially when it crashes a server. Earlier this year, reports began to surface about collections “ballooning” in size. Eventually a connection was drawn to using documents, but these were externally formatted documents, which means they are not stored inside a Helix collection. Internally stored documents do expand the size of a collection, but adding a link to a 200MB file stored externally shouldn’t cause the collection to grow by 200MB. Something was clearly wrong.

In one of those ‘unfortunately/fortunately’ scenarios, one of our best beta testers made a change to his main business collection that triggered this bug, and in one day his collection went from 800MB to 1.3GB! Unfortunately he had to turn off his new feature. Fortunately we now had somebody who could not only see the bug, but could report it to us with real examples of how to trigger it. Repeatable almost always means fixable, but the path is not always without a few twists and turns, and this was no exception.

We implemented a solution in Helix 6.1.9 that fixed the ballooning problem, but that the fix was making some collections less stable. So we took another run at the problem, and discovered that the true root of this problem was pretty deep and fundamental to the way records are stored and edited in Helix. In fact, the underlying problem could happen with any field type, not just documents.

So a new fix is now in place that should reduce the instances of ‘random’ crashing overall. It will become available in “Helix 6.1.10” before the year ends. Beta testers should get ready to get to work.

Helix 6.1.x is the last version that will support Classic Mac OS. Upgrading to Helix 6.1 is the best way to ease your transition to macOS, because there you can still use Helix RADE and Client in Classic on your older Macs, while testing the waters with macOS Engine and Client on your PowerPC and new Intel Macs. And joining the Europa Pioneer Program will let you do many of those things in RADE for macOS at the same time.

So if you are one of the holdouts who are still using pre-6.x Helix, waiting until Helix is completely converted to macOS, you should strongly reconsider. Helix 6.1 is so much more bug free than its predecessors that you owe it to yourself to take advantage of those improvements now.

And if you’re really waiting for RADE in macOS, stop waiting. It may not be “ready for prime time” yet, but it is here now and waiting for it to be complete will definitely not make it happen sooner. When Helix 6.2 is complete there won’t be a ‘transition’ to macOS; it will be all or nothing.

Finally, keep those collections humming, keep those upgrades coming, and help keep this train running. This leg of the journey is nearly done, and while running on fumes is nothing new for us, we don’t want to run out of wood before we run out of track.