I have searched the forums and FAQ's not finding what I want

Here is the dilemma I have.

I have T-Shirts for sale, range in size from Toddler to Adult XXXL with various colors (All Setup and functioning)

The Problem

We now need to adjust the Color Options with the Size Options

Meaning

For Youth Sizes Toddler too Youth Large I have only 4 color options

For Adults Sizes Adult Small to Adult XXXL I have now 8 Options

How do I set the Attribute / Options to only Display for the appropriate Size

Example
I need the drop down to only display the 4 colors when a Youth Size is picked
and need the all 8 colors to display when an Adult Size is picked.

Outside of Seperating Adult Size and Youth Size I am stumped

Please help, and thank you in advanced