zKB Import is broken for me

Please report bugs here, please be very detailed in your bug reports. If you've come up with some code to fix the bug, please post it here.
Post Reply
Bas
Apprentice
Posts: 65
Joined: Sun Apr 20, 2014 12:11
Location: Russia, Moscow
Contact:

zKB Import is broken for me

Post by Bas »

Code: Select all

[bas@server /.../kb/cron]$ /usr/local/bin/php /.../kb/cron/cron_zkb.php
2017/01/12 10:36:33 - Starting zKB Import
2017/01/12 10:36:44 - ZKBApi: https://zkillboard.com/api/allianceID/.../
2017/01/12 10:36:44 - 0 kills were posted and 400 were skipped (400 kills fetched)
2017/01/12 10:36:44 - Timestamp of last kill: 2016-12-10 20:46:13
2017/01/12 10:36:59 - ZKBApi: https://zkillboard.com/api/allianceID/.../
2017/01/12 10:36:59 - 0 kills were posted and 400 were skipped (400 kills fetched)
2017/01/12 10:36:59 - Timestamp of last kill: 2017-01-08 10:25:24
2017/01/12 10:36:59 - Time taken = 25.760802984238 seconds.
I tried:
- clear all cache
- disable all mods
- our logs show nothing: no errors in nginx.log (server and host), php-fpm.log
- apply patch to fix zkb fetcher (class.zkbfetch.php)
- run job from admin panel and from console

I don't understand, because zkb fetcher work on other killboards on my server and class.zkbfetch.php is same on all of them.
User avatar
Salvoxia
Developer
Posts: 1598
Joined: Wed Feb 22, 2012 12:11

Re: zKB Import is broken for me

Post by Salvoxia »

Hi,

do you get the same result if you set the last kill timestamp to a the current timestamp (to the hour)?
Since the timestamps you get are quite a bit in the past, is it not possible that all fetched mails are already present at your killboard?

Best Regards,
Salvoxia
Bas
Apprentice
Posts: 65
Joined: Sun Apr 20, 2014 12:11
Location: Russia, Moscow
Contact:

Re: zKB Import is broken for me

Post by Bas »

at the beginning: timestamp 2016-12-10 20:46:13

Code: Select all

ZKBApi: https://zkillboard.com/api/allianceID/.../
0 kills were posted and 400 were skipped (400 kills fetched). Timestamp of last kill: 2016-12-10 20:46:13
1st test: change timestamp from 2016-12-10 20:46:13 to 2016-12-11 00:00:00

Code: Select all

fetched some kills and timestamp return to 2016-12-10 20:46:13
2nd test: change timestamp to 2016-12-11 20:46:13

Code: Select all

ZKBApi: https://zkillboard.com/api/allianceID/.../
64 kills were posted and 136 were skipped (200 kills fetched). Timestamp of last kill: 2016-12-11 21:34:14
... and after that fetcher works fine
User avatar
Salvoxia
Developer
Posts: 1598
Joined: Wed Feb 22, 2012 12:11

Re: zKB Import is broken for me

Post by Salvoxia »

Okay, let me get this straight: During the first test, the last kill timestamp did not change at all?
Could you please PM me the alliance ID you fetched from? It still seems possible for the fetcher to get stuck under certain conditions.

Best Regards,
Salvoxia
Bas
Apprentice
Posts: 65
Joined: Sun Apr 20, 2014 12:11
Location: Russia, Moscow
Contact:

Re: zKB Import is broken for me

Post by Bas »

I try to change timestamp to 2016-12-11 01:00:00 and saw straight behavior: fetcher works fine, but timestamp move backward until 2016-12-10 20:46:13 and after fetcher stuck.

Besides I have stucked fetcher for friend alliance too, but with another timestamp.

/.../kb/cache/error.log show nothing, server logs too :(
Bas
Apprentice
Posts: 65
Joined: Sun Apr 20, 2014 12:11
Location: Russia, Moscow
Contact:

Re: zKB Import is broken for me

Post by Bas »

Okey. If I set timestamp to 2016-12-11 03:00:00 fetcher move forward. All early values - move backward.
User avatar
Salvoxia
Developer
Posts: 1598
Joined: Wed Feb 22, 2012 12:11

Re: zKB Import is broken for me

Post by Salvoxia »

Can you verify that you are using the latest version of zKBFetch (this one).
I tried your URL on my dev KB and it worked as expected: With increasing amounts of killmails posted within one hour, the amount of mails fetched also increased:

Code: Select all

90 kills were posted and 361 were skipped (600 kills fetched). Timestamp of last kill: 2016-12-10 20:54:46
90 kills were posted and 451 were skipped (600 kills fetched). Timestamp of last kill: 2016-12-10 21:05:19
90 kills were posted and 405 were skipped (600 kills fetched). Timestamp of last kill: 2016-12-10 21:30:49
2017/01/13 16:02:10 - 90 kills were posted and 652 were skipped (800 kills fetched)
2017/01/13 16:03:41 - 90 kills were posted and 743 were skipped (1000 kills fetched)
2017/01/13 16:04:55 - 90 kills were posted and 833 were skipped (1000 kills fetched)
2017/01/13 16:04:55 - Timestamp of last kill: 2016-12-10 23:07:39
2017/01/13 16:07:32 - 90 kills were posted and 594 were skipped (800 kills fetched)
2017/01/13 16:07:32 - Timestamp of last kill: 2016-12-10 23:32:41
Moving backward will occur, if a negative kill timestmap offset is set and your killboard does not already have the mails fetched prior to the start timestamp, and there are too many kills within that timeframe to post in one cycle. You either need to let it fetch all prior killmails until it moves forward again, or set the negative kill timestamp offset to 0.

Best Regards,
Salvoxia
Bas
Apprentice
Posts: 65
Joined: Sun Apr 20, 2014 12:11
Location: Russia, Moscow
Contact:

Re: zKB Import is broken for me

Post by Bas »

Salvoxia wrote: Fri Jan 13, 2017 20:34 Can you verify that you are using the latest version of zKBFetch (this one).
I checked this file and it has a different version.
I replace it with current version and problem solved.

Tnx again!
Post Reply