Each listing and the display for previous/next has its own setting ...

You can set the Previous Next for the product_info in the Configuration ... Product Info ...
Previous Next - Sort Order
Products Display Order by
0= Product ID
1= Product Name
2= Model
3= Price, Product Name
4= Price, Model
5= Product Name, Model
6= Product Sort Order