Upgrading Strategy Advice? ... for specialized site

[Feznizzle]

Hi,

I'm planning to deviate from the standard upgrade protocol, I'd appreciate it if anybody could poke holes in my plan.

MY PROPOSED STRATEGY (upgrading zc151 to zc154)
PART 1 – Prepwork
1. Create new subdomain (zc154.mysite), create new database, install fresh zc154.
2. Backup the live site db, import into fresh zc154, run the upgrade.
3. Install the responsive template.
4. Backup/import all the images from the live site.
5. Upgrade all mods, tweak until everything is running smooth.

Because time will have passed since Step2 above (upgrading Live site DB), I will need to manually capture any new customer data that may have occurred. So I will need to take the Live site db, upgrade it, dump the needed tables, and update the upgraded sites db. So...

PART 2 – Going live
1. Repeat Steps1&2 above on a new subdomain to get LIVE zc151 updated for zc154.
2. Track down and import all new traffic/customer data into the fully upgraded site db.
3. Port the upgraded site over to the Live domain.
4. Reinstall all modules (shipping, taxes, payment, etc) on Live.
5. Pray!


QUESTION

• Do you see any problems with this strategy?
• Can anybody tell me where I can find a list of tables that will change between db updating: address_book, orders, orders_status, admin_activity_log, and so on? If not, I can probably wing it. I may focus only on customers/orders because that is the most important bit and would limit the scope so I don't risk a screwup.
• Any thoughts? Suggestions to make this easier?

WHY DEVIATE FROM PROTOCOL?
This upgrade is about the toughest task I've ever faced (with web dev) and it's very much stretching my abilities.

I have three major problems. First, the site is heavily modded. Second, the site has tons of products, images, content (zipped, itweighs about 600mb). Third, the live version is active with new traffic, customers, orders, etc coming in every day.

One of the main goals I have is take the current template and upgrade it to a responsive version of itself. Since I'm trying to keep the same look for the primary version of the site (desktop), I need to work with all the categories/products/ezpages in place so I can match them up (style wise).

Because the site is so heavily modded (10+ addon mods, several of those have been heavily customized, plus there are few 100% custom mods), I have to upgrade the db numerous times as I upgrade the mods. I'm afraid that I'll forget critical database tweaks made subsequent to running various SQL updates. That could get super messy.

Plus, I'm very visual. I need to be able to see my progress and verify with my eyeballs that my changes are in place and working as they should. And the responsive version of the template is very different in some areas.

This just seems like the only way to go. For me.

What do you think?

[mc12345678]

My thought? Be prepared to take an extensive amount of time to get what you want.

Everything described can be accomplished with a single sql update file put together as you go along a single attempt of upgrading... you're far more likely to skip a critical step, object, piece of data following the above described deviation than to use the relatively tried and true method put together to provide an upgrade process applicable to the mass of people (thought I saw a number like tens of thousands?) that use ZC.

The above process as with the recommended process requires a moment of site outage in order to put things live while being up-to-date. So, that will be necessary regardless of the method. Further, from the above description the items of concern to "support" the newly upgraded site are things which you have total control over: product, category, etc. Description. These are things that can be updated via sql. The "hard" part may be to limit the live store product changes during the upgrade process, though those too could be captured in sql and then updated through the process...

Nothing described though seems like it necessitates the longer involved process of trying to correctly capture the differences in the two databases (at some point both database versions requiring an upgrade so that the information properly lines up).

Every online business tends to have some point in time when the access to the store is limited for a short duration. You can plan for this and notify your customers of an upcoming outage during which the final live upgrade would occur (file folder(s) moved, configure.php files modified as necessary or through the upgrade process, database upgraded with zc_install, sql patch file run for all the things that need to be captured as identified by doing the upgrade on the backup/sub-domain.)

My thoughts, asking for a lot of hurt... can be done, but the "incremental" modifications can be proven to yourself using the backup store and apply those in a single round rather than the outage that would exist to do each and every "upgrade" one at a time or try to individually recopy the changes made for the product(s) or other data to look as desired out the gate.

[torvista]

I must admit when I start to read the official instructions for upgrading my head starts to spin.

I have been upgrading to 155 for months but my process is.
Local install for development work, on php 7.
1) New Install of zc155a
2) Import real database
3) Upgrade database
4) Merge mods from old site but maintaining current template. Baby steps, and easy to see when something is out of place on the new site.
Write comments in any changes to core files.
In the process do a pile of spring cleaning and keep in mind the pending change of template.
This is what takes me ages, as I snarl at mods I did years ago that are too ugly to leave as is and spend time fixing stuff that isn't broken.
5) Make notes of any database fiddling required...when I find something that needs changing I backup the database and create/test an sql query to do the job automatically and so gradually build up a set of commands to use in one go when the time comes for the real upgrade.
6) When all seems well and good, replicate the files on the server like in /shop_155.
7)Import real database and upgrade again.
8)Apply custom sql changes.
9) Test to death.
10) Take old shop offline.
11) Import db to new shop and upgrade do sql commands again.
12) rename old /shop to /shop_154 for example.
13) rename new /shop_155 to /shop and change configure.php to suit.
14) Switch on.

I estimate 5-10 minutes offline, if that.

This leaves the old working shop in place as a reference.

When all seems ok, think about implementing new template (locally etc...).

[DrByte]

I do it slightly differently than torvista.

a) stage the whole thing on my own local server (new install, import db)
b) inspect all file differences, remaking all customizations into the new site, including upgrading of plugins
(Since I'm using a copy of the live db, I take notes of any new SQL changes I have to apply when upgrading plugins. Sometimes I apply those to the real live site before I actually go live, especially if I'm certain those changes won't negatively impact the live operation. This just helps minimize downtime when going live.)
c) test test test all along

When ready to go live:
d) copy the new /admin/ and /includes/ folders (and any other actual changed folders) to the live server as new_admin and new_includes
e) put the site down-for-maint
f) upload zc_install, and rename admin to old_admin, then new_admin to admin ... same with all other folders. This takes just a few seconds.
g) run zc_install to upgrade the db ... again just seconds to do
h) turn off down-for-maint

Done. Downtime is usually just a couple minutes. The critical part affecting shoppers is the swapping of the /includes/ folders from old to new, and the time spent with zc_install. Outside that shoppers aren't impacted, and no database data is lost or subjected to risk of damage/loss due to unnecessary export/import.


And I'd do template alterations as a completely separate step. That means I'd upgrade the present template by applying onto it any changes done to template_default between the versions. Then if a template change is needed it becomes a separate step. Again to minimize downtime.

[mc12345678]

While I would start with the template and torvista would add it later, regardless of that part (somewhat comes down to personal preference and that plugins are typically written to be functional with a base ZC install and therefore should continue to be functional with each other when properly incorporated to do so. But suggested part of all that, plan on creating a template directory. Then make the modifications desired/needed in it. This allows you to undo changes easily, to switch back to a default or even previously "functional" version of the template, etc... there is a plugin that is now available that supports backing up a modified template (expects all modifications to exist in the applicable template override directory).

This way, regardless of the final look (css, html tags) at least the operation(s) can be proven to function and when the template is actually applied, the "data" is there, and just have to work out the details.

I like the other way, such that as the new data is being added to the screen, other presentation modifications are made as necessary or the action location modified to get the desired result where wanted. Matter of taste for us and others. :)

[Feznizzle]

Thanks for the input, MC.

At the bottom, you say:

the "incremental" modifications can be proven to yourself -- in a single round rather than -- doing each -- "upgrade" one at a time

Good catch, sorry! I missplaced Step 4 under PART 2, should appear as Step 6 under PART 1.

Also you point that manually importing data can easily lead to data loss. To address that possibility, I've come up with this alternative. As I'm working PART 1 I will keep a diary of all sql updates. By that I mean, if an addon upgrade (say Ceon URI) requires that I run an sql update, I'll note it and set the updater aside. Then I'll run it later on the current db before going live.


Here is the revised plan:
PART 1 – Prepwork
1. Create new subdomain (zc154.mysite), create new database, install fresh zc154.
2. Backup the live site db, import into fresh zc154, run the upgrade.
3. Install the responsive template.
4. Backup/import all the images from the live site.
5. Upgrade all mods, noting any that require sql updates and setting the updaters aside.
6. Reinstall all standard modules (shipping, taxes, payment, etc).
7. Tweak until test site is running smooth.


PART 2 – Going live
1. Repeat Steps1&2 above on a new subdomain to get LIVE zc151 updated for zc154.
2. Rerun every sql update noted in Step 5 above.
3. Port the upgraded site files and db over to the Live domain.
4. Pray!

Is that better? Or do you still see the plan as problematic? Whatcha think, homey?

Btw, there will be no new prod/category/etc changes while i'm working. The only new data will be traffic related (new customers, orders, order updates, etc). But now that shouldn't matter, accept that if a new product is added the associated image will need to be re-uploaded (easy peasy).

[Feznizzle]

I made my last comment before realizing drbyte and torvista had commented.

I must admit when I start to read the official instructions for upgrading my head starts to spin.

AMEN!

Interesting that both of you say to upgrade the current template first. I'll have to think hard on that for a bit because my first instinct to go MC's route (start with template). Many (if not most) mods require template file alterations and overrides. Seems like a lot of extra work to have to do them all over again! But I can see how any problems with the upgrade would become obvious if you were looking at the site you already know so well.

Lot to process. Thanks so much for relaying your own methods.

[DrByte]

Many (if not most) mods require template file alterations and overrides. Seems like a lot of extra work to have to do them all over again!

Well, the amount of template-specific changes between ZC versions is generally minimal. So identifying and applying those changes usually takes a fraction of the time in my experience.


You said the "instructions make your head spin". I'm not sure why. They're just saying the same thing I posted above, but in a long detailed step-by-step way so the person who doesn't grok the terminology or concepts has something to go on.
I did abbreviate it quite a bit in this post awhile back: http://www.zen-cart.com/entry.php?3-...d-of-upgrading

[lat9]

One other thing. You've indicated that you're going to stage your upgrade in a subdomain of your site. Does your site employ a wild-card SSL certificate? If not, are you planning on purchasing an SSL certificate for that sub-domain?

FWIW, I usually stage an upgrade in a subdirectory off the target site (e.g. www.example.com/upgrade) so that I'm using the store's SSL certificate to validate the checkout-process and the SSL-side of the admin.

[Feznizzle]

lat9, that's an awesome suggestion! Thank you!!!

But... now I have a really, really dumb question to ask. If new/upgraded store is in a subdirectory, will it obey .htaccess rules from the folder above?

Drbyte, your point about easily finding changes in the older template makes a lot of sense.

Regarding the head spin bit, it's all the extra info added to aid grokking for the inexperienced that makes it painful. I actually took the extended version and copy/pasted into a blank file, then when thru and deleted all the grok stuff and anything that didn't apply or I didn't need to manually arrive at a simplified overview of the process, similar to what you posted above. :)