Twonky rescans (real scan) media store on every Win7 boot

General discussion about the media server. Feature requests. Hints, tips and tricks.
Locked
hawikoeln
Posts:7
Joined:Mon Aug 15, 2011 7:50 am
AV Hardware:Windows 7
Twonky rescans (real scan) media store on every Win7 boot

Post by hawikoeln » Mon Aug 15, 2011 8:06 am

Good morning. I searched the forum but couldn't find a question that describes exactly my problem so here we go:

* Twonky server is installed on Windows 7 32bit.
* My media is stored on an USB device (3TB) thats attached to this Windows box permanently.
* In Twonky config rescan is set to 0.

After every reboot of this Windows machine Twonky rescans the complete media store. In an older thread in this forum I could read that this is just a check and not a complete scan. On my machine this is not true. On the status page I can see that during this scan the count of media starts on 0 again. For me this looks like a complete rescan.

With nearly 300,000 prof images in hundreth of folders this process lasts for hours.

Is there an option within Twonky that tells Twonky not to rescan on reboot because the user knows about when the contents of his store changes?

User avatar
parnott
Posts:326
Joined:Mon Sep 20, 2010 9:55 pm
AV Hardware:Twonky, WDTV Live, XBMC (Linux & Windows)

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by parnott » Mon Aug 15, 2011 6:32 pm

Is there an option within Twonky that tells Twonky not to rescan on reboot because the user knows about when the contents of his store changes?
Not as far as I am aware.
After every reboot of this Windows machine Twonky rescans the complete media store. In an older thread in this forum I could read that this is just a check and not a complete scan. On my machine this is not true. On the status page I can see that during this scan the count of media starts on 0 again. For me this looks like a complete rescan.
I think you are confusing a rescan (which yes it does in my experience) with a complete database rebuild. A rescan is just validating the Twonky media database is consistent with the file system. A good thing in my opinion, there are other situations than when you change the content that could create inconsistencies (e.g. power failure, crash, etc.). A database rebuild rescans the shared files and completely rebuilds the database. Which will take some time. You can determine exactly what is happening by turning on logging and examining the log file.
With nearly 300,000 prof images in hundreth of folders this process lasts for hours.
Could be many factors here-
But how fragmented are the drives where the Twonky database is stored and where your media is stored?
When was the last time your de-fragmented these drives?

hawikoeln
Posts:7
Joined:Mon Aug 15, 2011 7:50 am
AV Hardware:Windows 7

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by hawikoeln » Mon Aug 15, 2011 10:38 pm

Thanks for your answer.

* This drive is new - bought just last week and was filled within 1 day. I don't think that fragmentation matters here.

* If the file counter starts on 0 on every reboot this does mean it's a complete rebuild I guess. If that is true what can I do to stop that?

Thanks

User avatar
parnott
Posts:326
Joined:Mon Sep 20, 2010 9:55 pm
AV Hardware:Twonky, WDTV Live, XBMC (Linux & Windows)

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by parnott » Tue Aug 16, 2011 12:03 am

When the counter starts at 0 on reboot, I don't think it is necessarily doing a rebuild. At least my experience is that Twonky shows how far into the rescan it is after a reboot, hence it starts at 0. The only real way to tell what is happening is to look at the log files.

But you said initially that on reboot that the rescan/rebuild (or whatever is happening) takes several hours. How do you know this? The only way I know to tell (apart from the logs) is if the USB disk is continously accessed for several hours or if the Twonkyserver process on your PC consumes a high % of cpu usage for several hours. Do you see this?

hawikoeln
Posts:7
Joined:Mon Aug 15, 2011 7:50 am
AV Hardware:Windows 7

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by hawikoeln » Tue Aug 16, 2011 7:37 am

Thanks again for your answer.

Just did boot the machine 20 minutes ago and now I'm looking at the web frontend of the Twonky Web UI from a second machine.

The counter is somewhere at 7,000 (of 350,000) and if I look at the "By Folder" listing I can see that it's at 'A' from 'A-Z' (my images are grouped in folders that start with a letter).

This means while this process (whatever it's called in the Twonky Universe) is running I can't access those files that were scanned every morning and are not scanned today yet. If those files are not accessible I bet it's a complete rebuild and not a rescan. And that is what I want to stop. Currently I have to wait for hours every day until all files are scanned and accessible in the Web UI. For me this sounds like a bug because Rescan is set to '0' in Config.

I will set logging on after this scan is complete and look at the result tomorrow morning.

User avatar
parnott
Posts:326
Joined:Mon Sep 20, 2010 9:55 pm
AV Hardware:Twonky, WDTV Live, XBMC (Linux & Windows)

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by parnott » Tue Aug 16, 2011 7:17 pm

Well what you are describing does seem like a database rebuild. Though I am at a bit of a loss to explain why. The only time I ever see this is when I explicitly delete the Twonky database and reboot. Twonky obviously has to rebuild the database in this situation.

Pure speculation, but maybe it has something to do with your files being on an external USB drive. In Windows, USB drives are flagged as removeable media, so maybe Twonky does something different with removeable media.

With regard to rescan=0, this has never applied at boot time. It is used after Twonky server is up and running to determine how/when rescans occur.

Ralf99
Posts:2
Joined:Mon Aug 22, 2011 6:37 pm
AV Hardware:Qnap

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by Ralf99 » Mon Aug 22, 2011 7:07 pm

I have the same problem here...
My System QNAP with Twonkyserver 6.0.34 or Version 5.xxx (is the same).

I have the rescan on "0" and EVERY Time i restart the nas, the Twonky Scan the whole Folder/ Files (>30.000 mp´s and >50.000 photos) ... the time they need is more than 2 Hours!! :(( thats really bad.

Is there any option to fix this??
(The existing database should be used and presented as it is. Why should anybody steal files while my NAS is off?)

Thanks cu Ralf

P.S. I have number 1) ... :roll:
[quote="parnott"]OK If you insist that it is rebuilding (rather than just rescanning) the database try these tests-

1) Go to the Twonky Config page and select Maintenance from the menu options on the left. Click on the 'Restart server' button at the top of the page. Wait a minute or so. Twonky will RESCAN all the files but it should only take a few minutes and you should be able to stream media during this time. This is what I see on my NAS.

2) Reboot your server. Wait a minute or so until the server is fully up and running. Try and access Twonky from your media player, if you can't and there is a lot of disk activity on the server that lasts more than a few minutes then yes Twonky is rebuilding its database because the database was removed during the reboot.[/quote]

User avatar
parnott
Posts:326
Joined:Mon Sep 20, 2010 9:55 pm
AV Hardware:Twonky, WDTV Live, XBMC (Linux & Windows)

Re: Twonky rescans (real scan) media store on every Win7 boo

Post by parnott » Mon Aug 22, 2011 10:14 pm

There is no option to change this behavior as far as I know.The scantime=0 option only comes into play after Twonky is up and running and validated the consistency of the file system and the database.

The problem is Twonky does not know how or why it was restarted. So it seems to do the file system rescan to get at least a minumum level of consistency between the media files and the database. There are a number of use cases that would suggest this is at least prudent-
1) System reboot, file system is unchanged, but Twonky does not know this but does a rescan
2) System reboot, file system is changed because file system check fixes/removes corrupt files/directories, I think you would want Twonky to do the rescan
3) Twonky is stopped (via telnet/ssh session), media files are changed, Twonky is restarted. Twonky does not know this happened so does a rescan

However, I have always run my Twonky server with scantime=15 and have never had any problems. I also don't turn off my server as I access the server at all odd times of the day/night because I use it as a general fileserver, not just for Twonky.

Locked