**Environment**
- Tainacan 1.4.1 (the same code is in `develop`, commit ab94a76…; the watchdog was introduced in 1.4.0)
- WordPress 7.1.3, shared hosting (cPanel)
- Site timezone: `America/Sao_Paulo` (UTC−3)
### Description
`Background_Process_Base::record_heartbeat()` stores `processed_last` in the **site's local time**:
```php
// src/classes/background-process/class-tainacan-background-process-base.php, line 659
[ 'processed_last' => current_time( 'mysql' ) ],
```
`detect_and_cleanup_stale_processes()` compares that value with a **UTC** cutoff:
```php
// same file, lines 604 and 614
$cutoff_date = gmdate( 'Y-m-d H:i:s', time() - $stale_timeout );
...
AND processed_last < %s",
```
On a site whose timezone is more than `$stale_process_timeout` (1800 s) behind UTC, a heartbeat written *right now* is already older than the cutoff. This includes every Brazilian timezone and most of the Americas. In UTC−3 the heartbeat looks 3 hours old, so the first watchdog run after a process starts marks it as stale.
Then:
1. The watchdog calls `close( $id, 'errored' )` (line 626).
2. `Background_Process::close()` re-queues the process (`status = 'waiting'`, line 215) and calls `dispatch()` immediately (line 237).
3. The watchdog calls `unlock_process()` (line 636), so the new request gets the lock.
4. The original worker is still alive and keeps going. Its loop in `handle()` never checks whether its process was re-queued.
The result is two workers processing the same batch range and writing to the same output.
Note that `Background_Process::update()` writes `processed_last` in UTC (`gmdate()`, `class-tainacan-background-process.php` line 151). The column therefore holds a mix of local and UTC values.
### Expected behavior
A process that is alive (it records a heartbeat on every iteration) must never be marked as stale. The watchdog should act only after `$stale_process_timeout` seconds without a heartbeat, regardless of the site's timezone, and a process should never run in two workers at the same time.
### Impact observed in production
The timestamps below are from the process logs, which are written with `gmdate()` (UTC). The site runs in pt_BR, so the messages appear translated. In the logs, "Processo enfileirado novamente (tentativa 1 de 3)" is "Process re-queued for retry (attempt 1 of 3)". In the error logs, "Processo marcado como falho pelo script de tentativas…" is "Process marked as errored by watchdog: no heartbeat within the stale timeout window."
**1. CSV export of a collection with 35,718 items (process 103)**
The export started at 00:25:30, and the watchdog fired at 00:26:00 and again at 00:28:40. The main log shows a "New Request" every ~20 s with no gap longer than 20 s, so the process was alive the whole time. Error log:
```
[2026-10-22 00:26:00] Processo marcado como falho pelo script de tentativas: nenhum sinal durante a janela de tempo.
[2026-10-22 00:28:40] Processo marcado como falho pelo script de tentativas: nenhum sinal durante a janela de tempo.
```
After the first re-queue, a second worker restarted from the last saved offset (item 1800) while the original one was at item 2900. The two workers then alternate batches. Abridged log, "Retrieving…" lines omitted:
```
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 2880
[2026-10-22 00:26:00] Processo enfileirado novamente (tentativa 1 de 3)
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 2900
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 2920
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 2940
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 1800
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 2960
[2026-10-22 00:26:00] Processing batch with 20 items, starting from item 1820
[2026-10-22 00:26:01] Processing batch with 20 items, starting from item 2980
[2026-10-22 00:26:01] Processing batch with 20 items, starting from item 1840
```
The batches starting at 1800 through 3700 (96 batches) were processed twice. The exported CSV has 37,638 data rows for 35,718 unique `special_item_id` values: 1,920 rows are duplicated, all with identical content. The process ends as "finished", and the UI shows "completed with errors".
**2. CSV import with 278 rows, all updates of existing items (process 104)**
The import started at 10:57:58. The watchdog fired at 10:58:02, four seconds later, and the second worker restarted from row 0 while the first one continued. Rows 0–223 were imported twice; because they were updates of the same values, no data was damaged. When the first worker finished, it deleted the source file, and the second one failed on row 224:
```
[2026-10-22 10:58:03] Processo marcado como falho pelo script de tentativas: nenhum sinal durante a janela de tempo.
[2026-10-22 10:58:22] Unable to read the source file. It may have been deleted, moved or made unreadable.
[2026-10-22 10:58:22] failed on item 224
[2026-10-22 10:58:22] Try Fatal Error: see the server error log for more details
[2026-10-22 10:58:22] Process aborted
[2026-10-22 11:00:26] Failed to delete importer source file.
```
The fix from #1265 does not cover this case. The file is deleted by the first worker, which closes normally as `finished`, not by an `errored` close.
### Steps to reproduce
1. Settings › General › Timezone: São Paulo. Any timezone more than 30 minutes behind UTC works.
2. Start a CSV export of a collection with a few thousand items.
3. Load any admin page while it runs, so WP-Cron fires the healthcheck.
4. Check the process log: "Process re-queued for retry (attempt 1 of 3)" appears while "New Request" lines keep arriving, and the error log shows the watchdog message.
With the timezone set to UTC, the same export runs without any retry.
### Reproduction with the 1.4.1 classes
I loaded the unmodified `Async_Request`, `Background_Process_Base`, `Background_Process` and `Background_Generic_Process` from tag 1.4.1, with a minimal WordPress stub that uses the real `current_time()` logic, over SQLite. Then I inserted a `running` process, called the real `record_heartbeat()` and invoked `detect_and_cleanup_stale_processes()`:
| Site timezone | `processed_last` after `record_heartbeat()` (UTC now 05:02:43) | Result |
|---|---|---|
| UTC | 05:02:43 | still `running` |
| Europe/Lisbon (UTC+1) | 06:02:43 | still `running` |
| America/Sao_Paulo (UTC−3) | 02:02:43 | `waiting`, retry_count 1, re-dispatched |
| America/Manaus (UTC−4) | 01:02:43 | `waiting`, retry_count 1, re-dispatched |
I also checked two production installs on the same server (Tainacan 1.4.1, WordPress 7.1.3, `America/Sao_Paulo`), with WordPress loaded through PHP CLI. On one of them, `current_time( 'mysql' )` returned `2026-10-23 02:41:21` while `gmdate()` returned `05:41:21`. The watchdog cutoff at that moment is `05:11:21`, so a heartbeat written right then was already 2.5 hours "older" than the cutoff.
The existing test `test_watchdog_does_not_touch_recent_processes()` does not catch this. It writes `processed_last` with `date()` (UTC under WordPress) instead of calling `record_heartbeat()`, and the suite runs with the default UTC timezone. Here is a test that fails on 1.4.1:
```php
public function test_watchdog_does_not_touch_live_process_with_negative_utc_offset() {
global $wpdb;
update_option( 'timezone_string', 'America/Sao_Paulo' );
$table = $wpdb->prefix . 'tnc_bg_process';
$wpdb->insert( $table, [
'data' => maybe_serialize( [ 'test' => true ] ),
'progress_value' => 0,
'user_id' => get_current_user_id(),
'priority' => 10,
'action' => 'generic_process',
'name' => 'Live Process',
'queued_on' => gmdate( 'Y-m-d H:i:s' ),
'status' => 'running',
'bg_uuid' => uniqid(),
'retry_count' => 0,
'max_retries' => 3,
] );
$id = $wpdb->insert_id;
$bg = new \Tainacan\Background_Generic_Process();
$bg->ID = $id;
$bg->record_heartbeat(); // a heartbeat written now
$method = ( new \ReflectionClass( $bg ) )->getMethod( 'detect_and_cleanup_stale_processes' );
$method->setAccessible( true );
$method->invoke( $bg );
$status = $wpdb->get_var( $wpdb->prepare( "SELECT status FROM {$table} WHERE ID = %d", $id ) );
$this->assertEquals( 'running', $status ); // 1.4.1: 'waiting' (re-queued)
update_option( 'timezone_string', '' );
}
```
### Suggested fix
1. Store the heartbeat in UTC, consistent with `update()`, `queued_on` and the watchdog cutoff:
```php
[ 'processed_last' => current_time( 'mysql', true ) ], // or gmdate( 'Y-m-d H:i:s' )
```
This also fixes the opposite effect on positive offsets. In UTC+9, for example, the watchdog currently acts 9 h 30 min after the last heartbeat instead of 30 min.
2. Hardening, as a separate issue if you prefer. A worker could confirm before each iteration in `handle()` that its process was not re-queued or closed by someone else, for example by re-reading `status` and `retry_count`, and stop if it was. Then neither a false positive nor a single task longer than the timeout could ever leave two workers writing the same output. Also, the watchdog's `unlock_process()` releases the lock of a worker that may still be alive.
### Workaround until a release
Add the negative UTC offset to the stale timeout (mu-plugin):
```php
foreach ( [ 'tnc-bg_import', 'tnc-bg_exporter', 'tnc-bg_generic_process' ] as $identifier ) {
add_filter( "{$identifier}_stale_process_timeout", function ( $seconds ) {
$offset = wp_timezone()->getOffset( new DateTime( 'now', new DateTimeZone( 'UTC' ) ) );
return (int) $seconds + max( 0, -$offset );
}, 1000 );
}
```
### Related
#1249 / #1265 dealt with the source file being deleted when the watchdog closes an import as `errored`. Part of the cases in #1249 may have had this cause. The workaround suggested there, raising `tnc-bg_import_stale_process_timeout` to 6 h, also hides this bug on UTC−3 sites, because 6 h is more than the 3 h offset.