Why on earth would you want to show only a random listing of part of the category's products instead of being able to see all of them?
Do you want to have the individual products look like the lower section instead of the upper section? That can be done in admin > Configuration > Product Listing. You can also set the number of products to show per page, and whether to show featured/new/etc. centerboxes.