🚀 go-pugleaf

Fundamental flaw in Java library distribution scheme
----------------------------------------------------

One of the major claims made in the Java white paper
(http://java.sun.com/1.0alpha3/doc/overview/java/index.html) and other
publications is that it will allow a true "Software IC" market to develop,
by solving the "third-party library" problem so that vendors can transparently 
distribute new versions of libraries without requiring recompilation 
of client programs (and as a corollary, that you don't need makefiles). 
In other words, it is claimed that Java solves the "fragile interface" and
"fragile superclass" problems, which are significant problems in C++.

This claim is not true.  After some investigation, it appears that programs
depending on this feature can behave incorrectly (i.e. not according to the
language specification).
In addition,
	- this faulty behavior occurs silently, and is not detected 
	  at runtime.
	- this behavior can violate interface encapsulation, so that
	  Java objects can not be relied on as secure 'capabilities' 
	  (in OS parlance), as claimed.  
  	- these problems are not fixable without significant changes to the 
  	  Java binary distribution standard.  The fixes required would 
  	  probably significantly increase the size of Java binary files (and
	  slow downloads). The changes required would also require breaking
	  upward compatibility with the beta release of the Java
	  distribution standard, which is claimed to be final and stable.


The Details
-----------

The problem, which can occur in several forms, is caused by an 
interaction between static typing and Java's signature-based runtime linking.
Consider the following example:

	class BankAccount {
		double savingsBalance;
		public BankAccount savingsDeposit(double amount) {
			savingsBalance = savingsBalance + amount;
			return this;
		}
	}

	class DeluxeAccount extends BankAccount {
		double checkingBalance;
		// ...
	}

Note that savingsDeposit() returns a BankAccount (perhaps other 
implementations might allocate a new account on every deposit, so that
the accounts are immutable and could be used in an audit trail).

Now suppose there is a method in a client library somewhere that
takes a DeluxeAccount and deposits some money into savings:
	...
		saveALittle(DeluxeAccount acct) {
			acct.savingsDeposit(1.00d);
		}

So far, everything is fine.  But suppose a new version of the bank account
library is distributed, that adds a new instance variable to keep track
of the total balance for deluxe accounts:
	...
	class DeluxeAccount extends BankAccount {
		double checkingBalance;
		double totalBalance;
		public DeluxeAccount savingsDeposit(double amount) {
			super.savingsDeposit(amount);
			totalBalance = totalBalance + amount;
			return this;
		}
		// ...
	}

To update the totalBalance, we have overridden the savingsDeposit method.
Note that we have specialized the return type to be DeluxeAccount (because
that is what the receiver is). 

If the saveALittle() method is executed (without recompiling) with this new 
library, a corrupted (wrong) totalBalance will result.  This occurs because
at runtime, the lookup process looks for a method with a signature that
EXACTLY matches the statically determined type of the method, for security
reasons.  Since the static type of the savingsDeposit() method was
BankAccount when saveALittle() was compiled, the overridden version of 
savingsDeposit() doesn't match, and is (erroneously) not found.  

Recompiling the compilation unit containing saveALittle() will produce a 
program that behaves differently, since the signature is different when it 
is recompiled.

The nasty thing about this is that the version of the method inherited
from BankAccount is NOT ACCESSIBLE to clients of a DeluxeAccount,
according to the language specification.  This means that supposedly, we
can count on the fact that there is no way the totalBalance could be
corrupted by clients of a DeluxeAccount.  This 'absolute' interface security
is the basis for the claim that Java objects can be used as
"capabilities".  But in the above example, object encapsulation
is violated.

Note that in this example, the signatures involved are for an overridden
method, where the only change in the signature is to a return type.
With some work, this could probably be fixed by changing the lookup
process to look for methods that have a return type that is the same OR
A SUBTYPE of the expected signature.  

Unfortunately, the problem has consequences that go further, and that
are far harder to fix.  A similar situation can occur when different argument
types are involved, which means that the methods are overloaded, instead of 
overridden.  In that case, it is not simply a matter of detecting signature
changes for a single method.  The problem here is that the client might
actually PICK A DIFFERENT METHOD ALTOGETHER if it was recompiled, based on
the overloading resolution rules.  In this case, the method signature
found at compile time is simply inadequate to find a matching method at
runtime.


Impact
------

Fixing this problem would require doing overload resolution at load time, 
which in addition to being slower, is not possible given the information
accessible in the (supposedly stable) Java binary distribution standard.
The exact static types of all actual arguments are required, which are 
not available at runtime (as opposed to the formal signature of the method 
invoked, which is what Java currently keeps).  This means that there is 
simply not enough information to solve the problem in the distribution 
format for Java.  Adding full actual argument type information to the 
binary form would allow runtime resolution to be done, but this would 
have a real size (and download delay) impact, since there are far more 
calls than there are method signatures called.

While it is marginally ok for decorative applets to "usually" behave
correctly, this is an unacceptable property if Java is to be used for
mission critical software.  It looks like some rethinking will have
to be done.