Zen Cart Logo
Forums / Bug Reports / [Done v1.3.9a] Customers Also Purchased - possible inefficient query

[Done v1.3.9a] Customers Also Purchased - possible inefficient query

Locked

Views: 1,947

Results 1 to 14 of 14
This thread is locked. New replies are disabled.
17 Mar 2010, 3:12 AM
#1
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

[Done v1.3.9a] Customers Also Purchased - possible inefficient query

This query keeps showing up in our slow query log... it is from the Customer Also Purchased area (was that a mod???)

Is there a way to make it more efficient? Each time a page is loaded, I see this in the log (times 3)

Or... is there a way to modify the original code to only look at the last XNumber of rows so it isn't examining so many rows??

thank you in advance :) :)

Query_time: 14 Lock_time: 0 Rows_sent: 3 Rows_examined: 196625

select p.products_id, p.products_image
from zen_orders_products opa, zen_orders_products opb, zen_orders o, zen_products p
where opa.products_id = '8664'
and opa.orders_id = opb.orders_id
and opb.products_id != '8664'
and opb.products_id = p.products_id
and opb.orders_id = o.orders_id
and p.products_status = 1
group by p.products_id
order by o.date_purchased desc
limit 3;

18 Mar 2010, 6:46 PM
#2
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

ok... this article gives the gist of what i believe is happening to us with this specific query...

http://hackmysql.com/case1

in the explain row we are always pulling over 194,000 rows to get 3 rows ... see the following ... how can this query be either reorganized or redone to not take so much in the way of server resources??

mysql explain...

id 1 select_type SIMPLE table o type ALL possible_keys PRIMARY key NULL key_len NULL ref NULL rows 194918 Extra Using temporary; Using filesort

id 1 select_type SIMPLE table opa type ref possible_keys idx_orders_id_prod_id_zen,opa.orders_id key idx_orders_id_prod_id_zen key_len 8 ref store_namehere.o.orders_id,const rows 1 Extra Using index

id 1 select_type SIMPLE table opb type ref possible_keys idx_orders_id_prod_id_zen,opa.orders_id key idx_orders_id_prod_id_zen key_len 4 ref store_namehere.opa.orders_id rows 3 Extra Using where; Using index

id 1 select_type SIMPLE table p type eq_ref possible_keys PRIMARY,idx_products_status_zen key PRIMARY key_len 4 ref store_namehere.opb.products_id rows 1 Extra Using where

18 Mar 2010, 8:00 PM
#3
wilt avatar

wilt

Oji-san

Join Date:
Jun 2003
Location:
Newcastle UK
Posts:
1,867
Plugin Contributions:
3

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

I would suspect (and from very cursory testing) that adding a separate index for products_id in the order_products table would help here.

e.g.

ALTER TABLE zen_orders_products ADD INDEX ( products_id )

If you are going to try this, please make sure you have a backup of your tables before running the alter statement.

18 Mar 2010, 9:22 PM
#4
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

That was so totally it... Thank you!! I was really unsure of where to add an index and your answer is spot on by the numbers I am getting now!! :)

Testing with the same query in my test area (why the rows examined would be different)...
Showing rows 0 - 2 (3 total, Query took 0.7324 sec) - before index 156767 rows_examined
Showing rows 0 - 2 (3 total, Query took 0.0087 sec) - after index 116 rows_examined

id 1 select_type SIMPLE table opa type ref possible_keys idx_orders_id_prod_id_zen,opa.orders_id,products_i... key products_id key_len 4 ref const rows 116 ExtraUsing temporary; Using filesort

id 1 select_type SIMPLE table o type eq_ref possible_keys PRIMARY key PRIMARY key_len 4 ref store_nameherebeta.opa.orders_id rows 1 Extra

id 1 select_type SIMPLE table opb type ref possible_keys idx_orders_id_prod_id_zen,opa.orders_id,products_i... key idx_orders_id_prod_id_zen key_len 4 ref store_nameherebeta.opa.orders_id rows 3 ExtraUsing where; Using index

id 1 select_type SIMPLE table p type eq_ref possible_keys PRIMARY,idx_products_status_zen key PRIMARY key_len 4 ref store_nameherebeta.opb.products_id rows 1 ExtraUsing where

18 Mar 2010, 9:46 PM
#5
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

I hate to say this... ack... but now our slow query log is adding up with the current bestsellers query. Maybe I should back out that index that I just added?

18 Mar 2010, 10:29 PM
#6
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

Leaving things the way they are for a bit :) (the index was put on the zen_orders_products table) now we have the current_bestsellers query running slow...

Information:

slow query log entry

Query_time: 11 Lock_time: 0 Rows_sent: 10 Rows_examined: 3381564

select distinct op.products_id, pd.products_name, sum(op.products_quantity) as total
from zen_orders_products op, zen_orders o, zen_products p, zen_products_description pd, zen_products_to_categories p2c, zen_categories c
where p.products_id = op.products_id
and pd.products_id = op.products_id
and o.orders_id = op.orders_id
and op.products_id = p2c.products_id
and p2c.categories_id = c.categories_id
and p.products_status = '1'
and p.products_price > '0'
and pd.language_id = '1'
and '63' in (c.categories_id, c.parent_id)
and TO_DAYS(NOW()) - TO_DAYS(o.date_purchased) <= 30
group by op.products_id
order by total desc, pd.products_name
limit 10;

ran in phpmyadmin - Showing rows 0 - 9 (10 total, Query took 11.1249 sec) (weird that the slow log shows over 3 million rows to examine, where phpmyadmin says 6406)

id 1 select_type SIMPLE table pd type ALL possible_keys PRIMARY key NULL key_len NULL ref NULL rows 6406 ExtraUsing where; Using temporary; Using filesort
id 1 select_type SIMPLE table p type eq_ref possible_keys PRIMARY,idx_products_status_zen key PRIMARY key_len 4 ref *****.pd.products_id rows 1 ExtraUsing where
id 1 select_type SIMPLE table op type ref possible_keys idx_orders_id_prod_id_zen,opa.orders_id,products_i... key products_id key_len 4 ref *****.p.products_id rows 56 ExtraUsing where
id 1 select_type SIMPLE table p2c type ref possible_keys PRIMARY,idx_cat_prod_id_zen key PRIMARY key_len 4 ref *****pd.products_id rows 3 ExtraUsing index
id 1 select_type SIMPLE table o type eq_ref possible_keys PRIMARY key PRIMARY key_len 4 ref *****.op.orders_id rows 1 ExtraUsing where
id 1 select_type SIMPLE table c type eq_ref possible_keys PRIMARY key PRIMARY key_len 4 ref *****.p2c.categories_id rows 1 ExtraUsing where

18 Mar 2010, 11:23 PM
#7
wilt avatar

wilt

Oji-san

Join Date:
Jun 2003
Location:
Newcastle UK
Posts:
1,867
Plugin Contributions:
3

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

I doubt very much that the new index would have any effect (in terms of slowing it down) on the best sellers query, so you are right in not backing it out.

I'll check the best sellers query and see what needs optimizing.

18 Mar 2010, 11:29 PM
#8
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

wilt:

ALTER TABLE zen_orders_products ADD INDEX ( products_id )

If you are going to try this, please make sure you have a backup of your tables before running the alter statement.

Okay, so, while wilt's suggestion is good and valid, it was initially intended as a test to see whether it would specifically resolve the problem reported.

The official change that's going into ZC core for v1.3.9 and newer is the following, which may even offer more efficiency:

ALTER TABLE orders_products ADD INDEX idx_prod_id_orders_id_zen (products_id,orders_id);
18 Mar 2010, 11:51 PM
#9
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

DrByte, we aren't running 1.3.9... would you suggest running yours at all? would that affect the other we already ran?
or...
just wait until we upgrade? :)

DrByte:

Okay, so, while wilt's suggestion is good and valid, it was initially intended as a test to see whether it would specifically resolve the problem reported.

The official change that's going into ZC core for v1.3.9 and newer is the following, which may even offer more efficiency:

ALTER TABLE orders_products ADD INDEX idx_prod_id_orders_id_zen (products_id,orders_id);

19 Mar 2010, 4:08 AM
#10
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

I know you're not running 1.3.9, because it's not been released yet.
If you want to test it, feel free. You can always remove it later (esp before doing the 139 upgrade) so as not to have duplicates.

19 Mar 2010, 2:50 PM
#11
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

LOL... have a chuckle on me ;)

Would I then have to drop or remove the previous index added?

DrByte:

I know you're not running 1.3.9, because it's not been released yet.
If you want to test it, feel free. You can always remove it later (esp before doing the 139 upgrade) so as not to have duplicates.

19 Mar 2010, 4:18 PM
#12
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

Once the new one is added, I would recommend dropping the other one, yes, because otherwise you'll have needless duplication.

23 Mar 2010, 1:25 PM
#13
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

Good Morning :)

We have managed to nip the slow query log (set to 10 secs) down to almost no more entries...
Catalog pages are loading consistently in about 2.5secs :) YAY!!

I do however have a new issue...

the admin is running slow... up to 15 seconds per page
nothing is registering in the slow query log for the admin side

Do you know where I can start looking?

23 Mar 2010, 1:29 PM
#14
ksoup avatar

ksoup

Zen Follower

Join Date:
Jan 2004
Posts:
413
Plugin Contributions:
1

Re: [Done v1.3.9a] Customers Also Purchased - possible inefficient query

More information...

Watching the cpu usage (we have 12gigRam - dedicated database server)
it keeps giving figures over 100%... 115.4 ... 123.7... etc.

side note: our loads are awesome! all under 1... the highest this week was a 7 with backup running....

it is painful to do anything in admin...

ksoup:

Good Morning :)

We have managed to nip the slow query log (set to 10 secs) down to almost no more entries...
Catalog pages are loading consistently in about 2.5secs :) YAY!!

I do however have a new issue...

the admin is running slow... up to 15 seconds per page
nothing is registering in the slow query log for the admin side

Do you know where I can start looking?