Application Initialization

Conventional systems ‘load’ an application when it is started. In old Unix systems this would typically be upon interpreting a command line naming the application. The load process would read a .o file pass over, the library references and modify them to match currently available libraries located at immutable addresses. Then the code of the application would take control and additional application specific acts would transpire.

As Keykos adapted IBM’s PL/I and H assembler for native use, we soon discovered that they both performed similar initializattion steps before they read the content of the task at hand. These apps were both designed before virtual memory was on the horizon or even a common concept. The application would learn how much memory it had available and subdivide most of that memory into blocks and initialize those blocks in some repetitive pattern and only then read the input file. Having virtual memory it was strategic for Keykos to provide the application a full 16 MB since only the actually required memory would be reified into real memory. The peculiar initialization patterns of these two apps thwarted this plan.

The solution was that upon building the app (not launching it) let the app have 16MB, let it initialize and then just as it came to read its input, capture its state, including the many initialized blocks and ‘compress’ them with an ad-hoc scheme that was designed with those specific simple patterns in mind. Upon application launch the compiler as it appeared post initialization was installed in a Keykos segment, and a physically small special segment was created that was compression of the 15MB initialized pattern.

As the compilation proceeded pages in that virtual segment would be touched. The resulting page fault would signal the decompressor, running outside the address space of the compiler, which would decompress that particular page and reify it.
As the compiler ran references to that special segment would be populated with real pages and any appropriate pattern as produced in the real initialization was reproduced, but just for that page. The compiler ran with its input exactly as it would have in the conventional system if it had been given 16MB.

The compiler and assembler had different but similar patterns that were each discovered and “compressed” upon new releases from IBM. Several such releases were thus accommodated. Without delving into the logic of the app were able in this particular cases to give the advantages of virtual memory to applications written without that in mind. Also note that the binding of common library references in the code, to shared libraries was done in the infrequent initialization step rather than in the frequent launch step.