Right, I've gone back a a few steps and reverted a few files back to their original versions. I've also included a couple of the edits from this thread to fix the two problems:
'out of stock items in the cart causing everything to be flagged out of stock' and also
'if not logged in the checkout button prompts to log in instead of looping back when items are out of stock'.

This leaves me pretty much where I started with just the one problem: my 'out of stock' attributed items are able to be checked out using the PayPal Quickpay button. The normal CHECKOUT button correctly loops back to the cart with an Out Of Stock message.

One thing I have learned through my recent problems that is still the case: if I increase amount of the out of stock item in the cart to more than the total stock of all attributes added together, it loops back correctly. So i believe it's testing the total stock level for the product, not the attribute stock level.

This edit suggested on page 70 of this thread still doesn't fix this problem for me.

****

Similar patch for making Pay Pal Express not allow checkout with not enough stock according to SBA.

Edit ipn_main_handler.php and find:

Code:
require('includes/application_top.php');

Just below that, add the following code:

Code:

// Stock Check - kiddo hack
if ( (STOCK_CHECK == 'true') && (STOCK_ALLOW_CHECKOUT != 'true') ) {
$products = $_SESSION['cart']->get_products();
for ($i=0, $n=sizeof($products); $i<$n; $i++) {

unset($attributes);
if(is_array($products[$i]['attributes'])){
$attributes = $products[$i]['attributes'];
} else {
$attributes = '';
}


if (zen_check_stock($products[$i]['id'], $products[$i]['quantity'],$attributes)) {
zen_redirect(zen_href_link(FILENAME_SHOPPING_CART));
break;
}
}
}

This will not make the yellow button un-active, but will send the customer back to the shopping cart so they can change the cart contents so no out of stock items.