You have a weird situation ...
You are using Priced by Attribute ... and you put the Product on Special, which is using the Priced by Attribute to compute the price ... but then, you say the Attribute itself (the lowest priced one which is the display price of the product) is not to have the Special applied to it ...
Now, if you were to mark that Attribute that does not get the Special Price as not to be included in the base price:
Include in Base Price
When Priced by Attributes NO
then the display price would be right and so would the price when added to the cart ... when the Special is applied it is now applied properly ...