Admin Session Times Out No Matter What Setting Change

[mikeel100]

Server Host: localhost (127.0.0.1)
Database: zc200_xxxxxxxx
Store Home: C:/xampp/htdocs/zencart-2.0.0_xxxxxxxxxx/
Server OS: Windows NT DESKTOP-xxxxxxx 10.0 build 19045 (Windows 10) AMD64
Server Date: 06/04/2024 12:14:03
Server Up Time: Unsupported
HTTP Server: Apache/2.4.54 (Win64) OpenSSL/1.1.1p PHP/8.0.23
PHP Version: 8.0.23 (Zend: 4.0.23)
PHP File Uploads: On
Upload Max Size: 512M
PHP Memory Limit: -1
POST Max Size: 1024M
Database Engine: MySQL 10.4.25-MariaDB
Database Host: localhost (127.0.0.1)
Database Date: 06/04/2024 12:14:02
Database Data Size: 488 kB
Database Index Size: 690 kB
MySQL Slow Query Log Status: Off
MySQL Slow Query Log File: XXXXXXXXXXXXXXXXXX.log
MySQL Mode: NO_ZERO_IN_DATE, NO_ZERO_DATE, NO_ENGINE_SUBSTITUTION
XAAMP V.3.3.0
Only my custom admin tools and UPS Shipping (New one) are installed

I have a 103,927 products that I am exporting (with my script) to a datafeed file and it takes around 45 minutes to and hour to finish.

I put set_time_limit(0); at the beginning of my script to disable max_execution_time and prevent log errors for max_execution_time exceeding 1800 seconds. This is the only way to get the script to run all the way through.

Still, I have to monitor admin and click on continue when the session time out warning pops up the first time at 30 minutes, then then every 900 seconds of processing.

I have changed Admin Session Time Out in Seconds from 900, 1800, then 3600 and changed the PA-DSS Admin Session Timeout Enforced setting to non-compliant.

I assume :-) the purpose of the admin settings are to disable that pop up warning. Am I wrong?

I grepped / searched all of the Zencart files for 900 and found that the sessions.php file is where most of the magic happens. But, I do not want to mess with this file since it is at the core of PA-DSS.

I changed php.ini max_execution_time to unlimited after it failed at 3600 too.

I made these changes in apache my.ini, [mysqld] max_allowed_packet=10M and [mysqldump] max_allowed_packet=160M

I added these to config_inc.php: $cfg['ExecTimeLimit'] = 3600; $cfg['LoginCookieValidity'] = 43200;

How do you stop the session from ending before the script finishes? I've done everything I know to disable the pop-up warning and timing out but to no avail.

Thanks.

P.S. can you increase your forum time out limit so people don't lose what work they have done to describe the issues they are facing? It's very frustrating but I have learned to copy my work before sending:-(

[DrByte]

Technically speaking, usually for each instance of a page load PHP runs in its own "thread" for just that "hit"/instance, and whatever timeout is set for it, as long as it's not contrary to some "master" configuration setting in php.ini that can't be overridden at runtime, it will actually run for that entire duration even if the browser page times out while waiting for a response.
But, it's possible that your local Windows version of PHP/Apache isn't set to be "threaded" like that. Adjusting that on Windows is not my area of expertise so I can't help with that.

That said, I offer the following:

- have you tried reworking your export script for better efficiency? 45 minutes for 100,000 products sounds excessive. Screams of missing db indexes that related to the queries being run, or missed opportunities to cache frequent queries or even remember already-retrieved data instead of looking it up again. Or queries that are adjusted to do optimized lookup joins on other tables all at once instead of doing lots of extra queries for each product being looped-over.

- you could try adjusting things so your script can be run from the command-line, like a cron job. I realize this is less intuitive in a Windows environment. The currency_cron script is one example.

- Don't tell anyone I told you this, but if you open your Admin's Whos-Online screen and click the option to refresh the page every 5 minutes, it'll keep reloading that page ... and if you're running that export script in another tab on the same browser it may remember the session and keep it open by virtue of the other one auto-refreshing.

[mikeel100]

Technically speaking, usually for each instance of a page load PHP runs in its own "thread" for just that "hit"/instance, and whatever timeout is set for it, as long as it's not contrary to some "master" configuration setting in php.ini that can't be overridden at runtime, it will actually run for that entire duration even if the browser page times out while waiting for a response.
But, it's possible that your local Windows version of PHP/Apache isn't set to be "threaded" like that. Adjusting that on Windows is not my area of expertise so I can't help with that.

That said, I offer the following:

- have you tried reworking your export script for better efficiency? 45 minutes for 100,000 products sounds excessive. Screams of missing db indexes that related to the queries being run, or missed opportunities to cache frequent queries or even remember already-retrieved data instead of looking it up again. Or queries that are adjusted to do optimized lookup joins on other tables all at once instead of doing lots of extra queries for each product being looped-over.

- you could try adjusting things so your script can be run from the command-line, like a cron job. I realize this is less intuitive in a Windows environment. The currency_cron script is one example.

- Don't tell anyone I told you this, but if you open your Admin's Whos-Online screen and click the option to refresh the page every 5 minutes, it'll keep reloading that page ... and if you're running that export script in another tab on the same browser it may remember the session and keep it open by virtue of the other one auto-refreshing.

Thanks for the insight Dr Byte.

The only index I have is pID as primary. I'll take a look at my queries to see if some other sort of indexing might help.

The reason it is taking so long, as I understand, I'm selecting from multiple zencart tables and left joining them for insertion into a temporary table, then cleaning up and extracting substrings from the descriptions for other fields in the temp table, and finally writing it to a tab delimited file with SELECT INTO OUTFILE.

Rather than try to explain every detail, I'll attach my script. What's in this file is actually part of my backup.php script...so you are only seeing what is in play for exporting datafeeds.

I will also try the "secret" you mentioned at the end...appreciate the tip.

[mikeel100]

4 Seconds for creating the temp table with 103,927 products...not bad. One of my other cleaning up queries is no doubt causing the slowdown.

I watched processes last night and it stayed on category tree building forever. I'm using Zencart's native function, zen_output_generated_category_path, for this part of my code. I built a MYSQL method for building the category tree then I remember Zencart has a function for tree building. So I converted to the Zencart way to see if it would speed things up...it didn't.

I probably should have put this in a forum question instead of a bug, my apologies. I'll stop posting now and figure something out.

DROP TABLE IF EXISTS  `build_datafeed`;
     
CREATE TABLE `build_datafeed`(
                `pID` INT(11) AUTO_INCREMENT,
                `mfg` VARCHAR(64) NULL DEFAULT NULL,
                `sku` VARCHAR(64) NULL DEFAULT NULL,
                `model` VARCHAR(64) NULL DEFAULT NULL,
                `name` VARCHAR(640) NOT NULL DEFAULT '',
                `desc` TEXT NULL DEFAULT NULL,
                `img` TEXT NULL DEFAULT NULL,
                `weight` FLOAT NOT NULL DEFAULT '0',
                `cost` DECIMAL(15, 2) NOT NULL DEFAULT '0.00',
                `map` DECIMAL(15, 2) NOT NULL DEFAULT '0.00',
                `msrp` DECIMAL(15, 2) NOT NULL DEFAULT '0.00',
                `shipping` VARCHAR(32) NULL DEFAULT '',
                `zip` VARCHAR(10) NULL DEFAULT NULL,
                `qty` FLOAT NOT NULL DEFAULT '0',
                `tags` TEXT NULL DEFAULT NULL,
                `length` VARCHAR(32) NULL DEFAULT '',
                `height` VARCHAR(32) NULL DEFAULT '',
                `width` VARCHAR(32) NULL DEFAULT '',
                `pqty` FLOAT NOT NULL DEFAULT '1',
                `cat` VARCHAR(64) NULL DEFAULT NULL,
                `catpath` TEXT NULL DEFAULT NULL,
                `added` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
                `freq` VARCHAR(64) NULL DEFAULT NULL,
                `lead` VARCHAR(64) NULL DEFAULT NULL,
                `eta` VARCHAR(64) NULL DEFAULT NULL,
                `updated` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
                `url` TEXT NULL DEFAULT NULL,
                PRIMARY KEY(`pID`)
        ) ENGINE = MyISAM AS SELECT
    `p`.`products_id` AS `pID`,
    `m`.`manufacturers_name` AS `mfg`,
    `p`.`products_model` AS `sku`,
    `p`.`products_model` AS `model`,
    `pd`.`products_name` AS `name`,
    `pd`.`products_description` AS `desc`,
`p`.`products_image` AS `img`,
`p`.`products_weight` AS `weight`,
`p`.`products_price` * 0.75 AS `cost`,
`p`.`products_price` AS `map`,
`p`.`products_price` AS `msrp`,
`p`.`ship_method`  AS `shipping`,
`p`.`ship_zip` AS `zip`,
`p`.`products_quantity` AS `qty`,
'' AS `tags`,
'' AS `length`,
'' AS `height`,
'' AS `width`,
`p`.`products_quantity_order_units` AS `pqty`,
`cd`.`categories_name` AS `cat`,
`cd`.`categories_id` AS `catpath`,
`p`.`products_date_added` AS `added`,
'' AS `freq`,
'' AS `lead`,
`p`.`products_date_available` AS `eta`,
`p`.`products_last_modified` AS `updated`,
'' AS `url`
FROM `products` `p`
INNER JOIN categories_description cd ON `p`.`master_categories_id` = `cd`.`categories_id`
INNER JOIN products_description pd ON `p`.`products_id` = `pd`.`products_id`
INNER JOIN manufacturers m ON `p`.`manufacturers_id` = `m`.`manufacturers_id`
INNER JOIN `categories` `c` ON `p`.`master_categories_id` = `c`.`categories_id`; 

[mikeel100]

@DrByte...dual admin Who's Online trick did stop the Session Timeout issue.

Thanks.