Chapter two of this thesis describes problems that have been encountered with interposing, or wrapping system calls in Unix in order to extend security patterns there.
In section 2.2 I learn that they are discussing kernel mode wrappers that runs after the transition to privileged mode and before the body of the legacy kernel code. His Precondition and Postcondition hooks correspond to Keykos wrapping patterns but ‘wrapping’ in the Keykos context has always implied being subject to capability enforcement.
The problem becomes clear in section 2.3. In the wrapper pattern under discussion the wrapper code accesses kernel state that is not locked and mutable by other processors. If the wrapper code locks the data, it must either be unlocked prior to giving control to the legacy code, or kept locked whereupon the legacy code will be fatally surprised to find it locked. If the wrapper unlocks the code another processor may change it before the legacy kernel code sees it. Bad pattern!! (as the author says.)
The Keykos kernel is designed for MP and the locks are essentially in place and already used for reasons beyond MP. The placing of lock and unlock instructions in the kernel code is a delicate matter and a wrapper trying to mesh with kernel locking strategy would not work! This note will give some insight why.
Wrapper patterns in Keykos are forced to respect abstractions and are presumably less efficient than the wrappers described here. They are also immune from the attacks described here.
I am in violent agreement with Chapter 2!
But one more thing:
Code that examines data provided by another module may generally assume that that data is unchanging during the examination. Occasionally code patterns are convenient that look at some of the data more than once—seldom is it necessary to consider the possibility that the data is changing. That is the eventuality that Watson arranged thus subverting the kernel’s charter. The problem does not necessarily disappear when the data examination has been limited to just one routine. It almost surely makes it less likely. Even the compiler may feel free to use two fetches where just one was expressed in the code but I can not think of a reason for it to do so. If the data in user space were declared volatile in the kernel and the program logic mentioned no byte more than once then no such exploit would be possible.
The arguments for not copying data upon crossing of protection boundaries diminish, especially for modest values like file names.