WatchFTP home page
WatchFTP Startseite (Deutsche Version)
WatchFTP Página Principal (Español)
Site WatchFTP (Français)
  Welcome, Guest. Please Login or Register
YaBB - Yet another Bulletin Board
  News:
WatchFTP v4 released. see http://www.watchftp.com/v4.html
  HomeHelpSearchLoginRegister  
 
Pages: 1
Send Topic Print
Updated File Timestamp == New File ? (Read 18264 times)
robertc
YaBB Newbies
*
Offline


Posts: 10

Updated File Timestamp == New File ?
Jan 30th, 2014, 3:38am
 
Is WF (3.5.2) supposed to detect UPDATED remote file ?
i.e file with newer timestamp only but unchanged name/size as existing one

Note: I DO NOT want WF to download. I have that checked. (so no local file to compare, so I guess that means WF has to keep track of file info between scans in order to detect the change...)
Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #1 - Jan 30th, 2014, 5:52am
 
Hello Robert,

Yes, see attached picture. Are you seeing something different?
Back to top
 

robert.png


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #2 - Jan 30th, 2014, 3:49pm
 
Gert wrote on Jan 30th, 2014, 5:52am:
Hello Robert,

Yes, see attached picture....


hi Gert,
But WF does NOT trigger "additional action", right ?

I did simple test:

Code:
touch existingfile.tst
touch newfile20131030.tst
 



Only the 2nd one (newfile20131030.tst) triggered action
Sad
Back to top
 
« Last Edit: Jan 30th, 2014, 7:08pm by robertc »  
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #3 - Jan 31st, 2014, 6:33am
 
WF does trigger the actions.

However (maybe this is what you were seeing), there is some code inside WF that will deliberately ignore timestamp changes in certain situations. I will have to check on that.

When you look at the "changes since" window, do you see the existingfile.tst as a changed file?
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #4 - Jan 31st, 2014, 6:40am
 
> When you look at the "changes since" window, do you see the existingfile.tst as a changed file?


Gert, no. only shows newfile20131030.tst.

BTW both newfile20131030.tst and existingfile.tst are zero byte.
Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #5 - Jan 31st, 2014, 7:14am
 
Can you right-click the task and select "backup", please email me the zip this creates.
You may want to change the user/pass in its settings first.
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #6 - Jan 31st, 2014, 7:51am
 
By the way, I just looked at the source code...

WF compares the "last modified" timestamps, so if your "touch" utility changes the "created" timestamp, that will not trigger WF.

Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #7 - Feb 1st, 2014, 9:18am
 
Gert wrote on Jan 31st, 2014, 7:51am:
By the way, I just looked at the source code...

WF compares the "last modified" timestamps, so if your "touch" utility changes the "created" timestamp, that will not trigger WF.



Gert,

OK then...what if the "touch" changes ALL 3:


[robert:~/xmldir]$uname -a
SunOS secdpdevdb01 5.10 Generic_144488-11 sun4v sparc SUNW,Sun-Blade-T6300

[robert:~/xmldir]$truss -v lstat -t lstat ls -l existingfile.tst
lstat64("existingfile.tst", 0xFFBFF718)         = 0
   d=0x04B40000 i=6485896 m=0100644 l=1  u=1133  g=600   sz=0
       at = Jan 30 11:23:25 EST 2014  [ 1391099005.118259000 ]
       mt = Jan 30 11:23:25 EST 2014  [ 1391099005.118259000 ]
       ct = Jan 30 11:23:25 EST 2014  [ 1391099005.118270000 ]
   bsz=8192  blks=0     fs=ufs
-rw-r--r--   1 robert    acusers       0 Jan 30 11:23 existingfile.tst

[robert:~/xmldir]$touch existingfile.tst

[robert:~/xmldir]$truss -v lstat -t lstat ls -l existingfile.tst
lstat64("existingfile.tst", 0xFFBFF798)         = 0
   d=0x04B40000 i=6485896 m=0100644 l=1  u=1133  g=600   sz=0
       at = Feb  1 02:46:21 EST 2014  [ 1391240781.811244000 ]
       mt = Feb  1 02:46:21 EST 2014  [ 1391240781.811244000 ]
       ct = Feb  1 02:46:21 EST 2014  [ 1391240781.811254000 ]
   bsz=8192  blks=0     fs=ufs
-rw-r--r--   1 robert    acusers       0 Feb  1 02:46 existingfile.tst




History: I have done several "touch existingfile.tst" during this:



Changed since:




Are you not able to reproduce what I'm observing ?

I mean are you actually seeing that each of YOUR "touch existingfile.tst" between each scans trigger the action ?







Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #8 - Feb 1st, 2014, 9:54am
 
I just used FileZilla:
* downloaded a file in binary mode (so the contents of the file do not change)
* the local file now has all its timestamps set to the current date/time
* uploaded the unchanged file in binary mode
WatchFTP detected it fine and downloaded the file.

Back to top
 
« Last Edit: Feb 2nd, 2014, 8:36am by Gert »  


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #9 - Feb 1st, 2014, 9:58am
 
By the way, WF has a setting somewhere "ignore zero byte files", perhaps that option is "on" for your task?
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #10 - Feb 1st, 2014, 7:49pm
 
Gert wrote on Feb 1st, 2014, 9:54am:
I just used FileZilla:
* downloaded a file in binary mode (so the contents of the file do not change)
* the local file now has all its timestamps set to the current date/time
* uploaded the unchanged file in binary mode
WatchFTP detected it fine and downloaded the file.



Gert thanks for checking it out but please let me stress this again as I did in post#1 and please take note:

Quote:
robertc Posted on: Jan 30th, 2014, 3:38am

Note: I DO NOT want WF to download. I have that checked. (so no local file to compare, so I guess that means WF has to keep track of file info between scans in order to detect the change...)


Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #11 - Feb 1st, 2014, 7:54pm
 
WF 3.5 (and higher) does not compare timestamps to a local file. That was the big change for 3.5. It maintains a little db with timestamps and sizes of files and compares to that.

In my previous post I just mentioned "download" because that is what my standard test task does. It should work fine in your case when you do not download.

Please try the link to the beta I posted earlier, that was the version I used with my test.

Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #12 - Feb 1st, 2014, 8:45pm
 
Quote:
It maintains a little db


SQLite right ? yes I have see reference to it in one of WF's txt files
I have used it as embedded db in my own software.


I will try the beta, so do I have to re-create the tasks after install ?
Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #13 - Feb 1st, 2014, 8:50pm
 
the history.db is sqlite.

The timestamps/sizes are inside ftpdirlist_yyyymmdd.watchftp files inside the task directory. Plain txt, but please do not try to change them, the format is a little special and it may cause aborts. The changes since window basically shows the diff of 2 of those files.

Just install "on top" of your current install and all settings/tasks will work fine.
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #14 - Feb 2nd, 2014, 12:34am
 
Gert,

Tried 3.6.0 - Same thing, no change, no action triggered, blank "Change since" - just like before.

I really think for the sake of good customer service, you should just pretend you're QA hired by Gert and not the programmer - simply create a task from scratch,  configured the same as what I have - which I think is about as simple as it gets - and see if you have the same issue I'm seeing.

Hope I don't come thru as a jerk but I have spent too much time on this rather trivial thing.

Here I even recorded my task settings: http://screencast.com/t/GUBsZVmQsCOS


thanks
Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #15 - Feb 2nd, 2014, 7:18am
 
I have created a new task based on your settings. The only special thing I saw with your settings is that you use SFTP. SFTP and FTP are 2 totally different protocols, even if they share the "ftp" in their name, they work totally different.

Created the task, let it do an initial scan (so all current file sizes and timestamps are recorded without taking action), used FileZilla again to download/re-upload a file so its timestamps are touched.

The file was detected by WF and my bat file triggered. What you can try, on the FTP Server Settings tab, is check the option "keep session log". This will record all activity in a yyyymmdd.log file (or right-click the task and "view session log"). Run the task and (after touching existingfile.txt) open the log. Search for all occurrences of existingfile.txt and you should some filesize/timestamp info for the file.

For example, I "touched" the file "dfsgsddfz- kopie.txt", and see the following in the log:

20140202 064253 ** found file dfsgsddfz- kopie.txt
20140202 064253              Size 5005
20140202 064253        Last Modif 30-8-2012 11:47:08
20140202 064253           Created 30-8-2012 11:47:08
...
20140202 064723 ** found file dfsgsddfz- kopie.txt
20140202 064723              Size 5005
20140202 064723        Last Modif 2-2-2014 6:46:57
20140202 064723           Created 2-2-2014 6:46:57

and my bat file triggered fine. I now see the logging code doesn't log the fact it will trigger actions in this case, I will add that to the code.

My bat script is really simple, it just echos all environment variables:
Code:
SET WF_ > f:\testing\output.log
 


and after running the task, this is what I see in my output.log file
Code:
WF_AGECREATED_DAYS=
WF_AGECREATED_HOURS=
WF_AGEMODIFIED_DAYS=
WF_AGEMODIFIED_HOURS=
WF_CONFIG=RCTEST
WF_CUREVT=1391320042
WF_CURRUN=1
WF_DATE=20140202
WF_DIR=
WF_DTTM=20140202 064752
WF_FILE=C:\TaskDoesntDownload\home\watchdir\WATCHFTPTESTS\dfsgsddfz- kopie.txt
WF_FILE_B=dfsgsddfz- kopie
WF_FILE_D=C:\TaskDoesntDownload\home\watchdir\WATCHFTPTESTS\
WF_FILE_DATE_CR=
WF_FILE_DATE_LM=
WF_FILE_E=TXT
WF_FILE_LOCAL=C:\TaskDoesntDownload\home\watchdir\WATCHFTPTESTS\dfsgsddfz- kopie.txt
WF_FILE_N=dfsgsddfz- kopie.txt
WF_FILE_REMOTE=/home/watchdir/WATCHFTPTESTS/dfsgsddfz- kopie.txt
WF_FILE_SHORTNAME=
WF_FILE_SHORTPATH=
WF_FILE_SIZE=
WF_FILE_SIZE_KB=
WF_FILE_TIME_CR=
WF_FILE_TIME_LM=
WF_FIRSTDIR=:
WF_INSTDIR=C:\Program Files\WatchFTP
WF_OFILE=
WF_OFILE_B=
WF_OFILE_D=
WF_OFILE_E=
WF_OFILE_N=
WF_ORELDIR=
WF_PARENTDIR=WATCHFTPTESTS
WF_REASON=FILENEW
WF_RELDIR=:\TaskDoesntDownload\home\watchdir\WATCHFTPTESTS
WF_TASK=RCTEST
WF_TASKDIR=C:\ProgramData\WatchFTP\RCTEST
WF_TIME=064752
 



Note that I asked you for a task backup earlier so I could properly look at your settings and recreate the task. Sorry to hear you don't appreciate 7-days per week service from the one person in my company that knows most about the working of our programs.
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #16 - Feb 2nd, 2014, 7:32am
 
I think I have found the issue. I could recreate the issue when I have the (disabled) option "Only download new files" checked.

On the Synchronization tab, please do the following
- UNCHECK "Do NOT download detected files"
- UNCHECK "Only download new files"
- CHECK "Do NOT download detected files" again.

This is clearly a bug, introduced in v3.5 when "do not download" was introduced. When "do not download" is selected, WF should ignore the "only download new files" setting. This will be fixed in the upcoming 3.6 release.

Alternatively, download the new beta: http://www.watchftp.com/cgi-bin/yabb2/YaBB.pl?num=1391325737
Back to top
« Last Edit: Feb 2nd, 2014, 8:37am by Gert »  

robert_001.png


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #17 - Feb 2nd, 2014, 9:21pm
 
Thanks, will try it out

So "new files" in "Only download new files" means file(s) with non-existing name(s) ?
Back to top
 
 
  IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #18 - Feb 2nd, 2014, 10:13pm
 
Gert,
Thanks !

Per your instruction, it's working as expected now, each "touch existingfile.tst" is detected.

I have also done "touch existingfile2.tst", "touch existingfile3.tst" between scans, notice that the Original Timestamp column is blank:

Is this supposed to be so ?



BTW, I used SFTP because I have to Smiley
(there's a freeware that works like WF but does not support SFTP)

Back to top
 
 
  IP Logged
robertc
YaBB Newbies
*
Offline


Posts: 10

Re: Updated File Timestamp == New File ?
Reply #19 - Feb 2nd, 2014, 10:33pm
 
Gert

Question:
Can I use the 2nd Batch Script option "Run after multiple downloads" in my case where I do not want to download ?

Back to top
 
 
  IP Logged
Gert
YaBB Administrator
Translator
*****
Offline


Posts: 441
The Netherlands
Re: Updated File Timestamp == New File ?
Reply #20 - Feb 3rd, 2014, 11:55am
 
Yes, that is possible.

Note that the only variables available are
WF_CONFIG=RCTEST
WF_REASON=AFTERDOWNLOADS
WF_TASK=RCTEST


About the "missing" original timestamp..... The window compares the current state to yesterday. Yesterday the file didn't exist so it shows "new file" without original info (which it does for changed files).
Back to top
 
 


Contact me at support -at- gdpsoftware -dot- com
My blog: http://blog-en.gdpsoftware.com/
WWW   IP Logged
Pages: 1
Send Topic Print