6.01 Not Parsing iTunes Playlists Correctly

General discussion about the media server. Feature requests. Hints, tips and tricks.
Locked
HK-47
Posts:4
Joined:Wed Aug 25, 2010 7:09 am
AV Hardware:QNAP TS-459 Pro running SUSE Linux
6.01 Not Parsing iTunes Playlists Correctly

Post by HK-47 » Wed Aug 25, 2010 7:32 am

Twonkyserver version: 6.01 Linux x32
iTunes 9.2.0.61

My iTunes playlists show up in the Twonky Server Media Browser. However, when clicking on the playlist names there are not any songs in them. After some investigating I've found that Twonky is reading my iTunes Library.xml file but not setting the correct path to the songs.

Here's a sample from my iTunes Library.xml file:
<key>2315</key>
<dict>
<key>Track ID</key><integer>2315</integer>
....
<key>Persistent ID</key><string>9F325AC53B5E2AD2</string>
<key>Track Type</key><string>File</string>
<key>Location</key><string>/share/Matt/Music/iTunes/Music/Compilations/The%20Big%2080's/Take%20On%20Me.m4a</string>
<key>File Folder Count</key><integer>5</integer>
<key>Library Folder Count</key><integer>1</integer>
</dict>


Here's the Twonky file music/"uploaded songs"/iTunes-playlist-80_s_Songs##M.wpl for my "80's Songs" playlist:
<?wpl version="1.0"?>
<smil>
<head>
<meta name="Generator" content="TwonkyMedia"/>
<meta name="ItemCount" content="17"/>
<meta name="Type" content="M"/>
<meta name="Source" content="iTunes"/>
<title>80's Songs</title>
</head>
<body>
<seq>
<media src="es/Music/Compilations/The Big 80's/Take On Me.m4a"/>
<media src="es/Music/Compilations/Billy Idol_ Greatest Hits/Dancing With Myself.m4a"/>
<media src="es/Music/Compilations/Billy Idol_ Greatest Hits/White Wedding, Pt. 1.m4a"/>
<media src="es/Music/Compilations/The Big 80's/Rapture.m4a"/>
<media src="es/Music/Compilations/The Big 80's/You Might Think.m4a"/>
<media src="es/Music/Chris Isaak/Best of Chris Isaak (Remastered)/Wicked Game 1.m4a"/>
<media src="es/Music/Eurythmics/Eurythmics_ Ultimate Collection/03 Sweet Dreams (Are Made of This) [.m4a"/>
<media src="es/Music/Compilations/The Big 80's/Cars.m4a"/>
<media src="es/Music/Compilations/The Big 80's/We Got The Beat.m4a"/>
<media src="es/Music/Men at Work/Men At Work_ Super Hits/Down Under 1.m4a"/>
<media src="es/Music/Compilations/The Big 80's/The Safety Dance.m4a"/>
<media src="es/Music/Compilations/The Big 80's/99 Red Balloons.m4a"/>
<media src="es/Music/Nena/99 Luftballons/11 99 Luftballons.m4a"/>
<media src="es/Music/New Order/Singles/Blue Monday 1.m4a"/>
<media src="es/Music/Compilations/The Big 80's/Abracadabra.m4a"/>
<media src="es/Music/Compilations/The Big 80's/867-5309_Jenny.m4a"/>
<media src="es/Music/UB40/The Very Best of UB40, 1980-2000/Red Red Wine.m4a"/>
</seq>
</body>
</smil>
</body>
</smil>
sic/Compilations/The Big 80's/Abracadabra.m4a"/>
<media src="e/Matt/Music/iTunes/Music/Compilations/The Big 80's/867-5309_Jenny.m4a"/>
<media src="e/Matt/Music/iTunes/Music/UB40/The Very Best of UB40, 1980-2000/Red Red Wine.m4a"/>
</seq>
</body>
</smil>
40/The Very Best of UB40, 1980-2000/Red Red Wine.m4a"/>
</seq>
</body>
</smil>

Somehow the path is being changed from /share/Matt/Music/iTunes/Music/Compilations/ to es/Music/Compilations/. The last part of the *.wpl file is all messed up too. This looks like a bug.

twonky
Posts:2
Joined:Tue Dec 09, 2008 7:22 am
AV Hardware:Pioneer PDP 111DD, Linux server, MacBook Pro Core 2 Duo

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by twonky » Tue Oct 26, 2010 6:44 pm

I am running Twonkyserver 6.0.23 on Ubuntu and I am feeding it my Music folder from my Mac (the same path, I created a /Users folder on the server and put my music folder in the same path as it is on my Mac so the paths should match). It is failing to pick up the Playlists I have on iTunes. Is this a know issue and is there a workaround/fix for this?

Thanks,
Frank

Wires
Posts:44
Joined:Mon Jan 25, 2010 2:16 am
AV Hardware:PS3

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by Wires » Fri Nov 26, 2010 2:20 am

I've had this issue since 5.x.x.

Oddly though, some of my playlists work, some do not. I don't know why it's being selective.

Twonky is showing the proper number of music files in each playlist, but the ones that don't work come up blank.

HK-47
Posts:4
Joined:Wed Aug 25, 2010 7:09 am
AV Hardware:QNAP TS-459 Pro running SUSE Linux

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by HK-47 » Mon Nov 29, 2010 2:34 am

I just upgraded to Twonkyserver version 6.0.28 on Linux and I still have this problem. The playlist names show up and they show the correct number of songs, but when you click on the playlist it's empty.

As I stated before this is a bug with the Twonky iTunes playlist plugin (/usr/local/twonkymedia/plugins/itunes-import). The beginning of the path in the generated iTunes-playlist-xxx.wpl files gets truncated so Twonky can't find the song when you go to play it. Manually editing the *.wpl files to correct the path allows you to play the songs in your playlist again but every time you restart the server these files get recreated and messed up again.

I tried submitting a bug report to Twonky support but someone in broken english blew me off saying that I installed the program wrong. After providing more technical information and asking for my ticket to be forwarded to a programmer I didn't receive any further responses.

I could write my own Perl script to replace their broken script but I shouldn't have to do that for software I paid for. This is an easy bug to fix.

Wires
Posts:44
Joined:Mon Jan 25, 2010 2:16 am
AV Hardware:PS3

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by Wires » Tue Nov 30, 2010 3:16 am

I'm running this on windows, so I went and looked for the WPL files you indicated. When I open them, the correct path is listed, but none of the itunes playlists will play (like you mentioned, correct # of items in the list, but blank).

I found the reason I had some playlists, was I had made a bunch of .M3U playlist files, and Twonky was parsing and listing them as well.

But none of the itunes ones play. Even with the right path in the .WPL file....

Wires
Posts:44
Joined:Mon Jan 25, 2010 2:16 am
AV Hardware:PS3

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by Wires » Tue Nov 30, 2010 4:22 am

Here is a copy of one of my itunes playlists. As you can see, the path is all correct. I can copy paste that path into Windows Media Player or a browser window, and it plays. But when I go to Twonky, the On-The-Go 3 playlist says "1" item, but it's blank.....

If I copy/paste M:\Music\Shinedown\Us and Them\02-Shinedown - Heroes.mp3 into a text file, save it as .M3U, Twonky parses it and gives me the playlist. So there is obviously something screwed with the itunes support.



<?wpl version="1.0"?>
<smil>
<head>
<meta name="Generator" content="TwonkyMedia"/>
<meta name="ItemCount" content="1"/>
<meta name="Type" content="M"/>
<meta name="Source" content="iTunes"/>
<title>On-The-Go 3</title>
</head>
<body>
<seq>
<media src="M:\Music\Shinedown\Us and Them\02-Shinedown - Heroes.mp3"/>
</seq>
</body>
</smil>
</body>
</smil>

Wires
Posts:44
Joined:Mon Jan 25, 2010 2:16 am
AV Hardware:PS3

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by Wires » Tue Nov 30, 2010 5:13 am

Ok, did a little experiment. Uninstalled Twonky, searched the drive for anything twonky related, and deleted it.

Reinstalled twonky, and after an hour of it parsing the shared folders, I now have itunes playlists again (On windows)

Important note though: Just uninstalling twonky won't do it. I've did that for every version change, and it's never worked to get my itunes playlists. You need to search for "twonky" and delete any folders relating to it.

HK-47
Posts:4
Joined:Wed Aug 25, 2010 7:09 am
AV Hardware:QNAP TS-459 Pro running SUSE Linux

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by HK-47 » Wed Dec 08, 2010 8:34 am

Ok, I wrote my own iTunes playlist parsing script. It works fine on my Linux NAS but I don't have any other systems to test it with. Technically, it should work on Windows and Mac too. Here are the instructions for Linux:
1) Install the Perl scripting language (most Linux distros install this by default)
2) Install the File::Slurp module from CPAN. You can download it here: http://search.cpan.org/~drolsky/File-Sl ... e/Slurp.pm (install instuctions are in the README inside the gzipped tar file)
3) Rename the original file /usr/local/twonkymedia/plugins/itunes-import to itunes-import.twonky
4) Place the code below into a new file /usr/local/twonkymedia/plugins/itunes-import. Make sure this new file has the same owner/group/permissions as the original file. Also, you may want to enter a fresh carriage-return after the first line in the script to make sure it runs. Edit the config section at the top of the script.
5) Add the following line to your twonky server startup script (mine is in /etc/init.d) in the "start" case just before "$TWONKYSRV -D 2>&1 >> /var/log/twonkymedia.log":
/usr/local/twonkymedia/plugins/itunes-import
6) Restart the Twonky server

BEGIN CODE ############################
#!/usr/bin/perl

##################################################################
# itunes-import Version 1.0
# Updated: December 7, 2010
# Programmer: HK-47
##################################################################
# This script reads the iTunes Library.xml file, and creates the playlist files for
# the Twonky media server.
##################################################################

use File::Slurp;

# The iTunes Library.xml file that iTunes uses (use full path)
my $ITUNES_LIBRARY = '/share/Matt/Music/iTunes/iTunes Library.xml';

# Location of Twonky iTunes playlist *.wpl files (use full path without trailing slash)
my $TWONKY_WPL = '/share/twonkymedia/media/music/uploaded songs';

# List of iTunes playlists you want to ignore
# (the script will NOT generate Twonky playlists for these)
my @IGNORE = ('Library', 'Genius', 'Books');

# This is a list of mapped drives or network shares that iTunes uses that we
# need to replace with the full path on the system where the files are located
# Note: no trailing slashes
my %PATH_SUBS = (
'file://localhost//GALACTICA' => '/share',
'file://localhost/M:' => '/share/Matt/Music'
);

# File owner for generated .wpl files
my $FILE_OWNER = 'twonky';

# File group for generated .wpl files
my $FILE_GROUP = 'twonky';

# File permissions for generated .wpl files
my $FILE_PERMS = 0640;

##################################################################

# Read the iTunes Library.xml file
# (note: This method slurps the whole file in at once for speed.)
my $r_iTunes = read_file( $ITUNES_LIBRARY, scalar_ref => 1 );

# Substitute the full path for mapped drives or network shares
foreach my $path_to_sub ( keys(%PATH_SUBS) ) {
$$r_iTunes =~ s/$path_to_sub/$PATH_SUBS{$path_to_sub}/g;
}

# Create a hash that stores the track id and path
my %tracks = $$r_iTunes =~ m|<key>Track ID</key><integer>(\d+)[\w\W\s]*?<key>Location</key><string>(.+)<|g;

# Parse the playlist data and store it in a hash of arrays
# $playlists{'name'} = [track id,...];
my %playlists;

# Get to the playlist section of the string
$$r_iTunes =~ m|Playlists|g;

# The outer loop grabs the name of each playlist
while ( $$r_iTunes =~ m|\G[\w\W\s]*?<key>Name</key><string>(.+?)</string>|og ) {
my $pl_name = $1;
$playlists{$pl_name} = [];

# This inner loop gets the list of Track ID's for this playlist
while ( $$r_iTunes =~ m|\G([\w\W\s]*?)</dict>|og ) {
my $offset = pos($$r_iTunes);

if ( $1 =~ m|<key>Track ID</key><integer>(\d+)|o ) {
push( @{$playlists{$pl_name}}, $1); }
else {
# We're done with this playlist
pos($$r_iTunes) = $offset;
last;
}
}
}

# Create a .wpl file for each playlist
foreach my $playlist ( keys(%playlists) ) {
# Should we ignore this playlist?
if ( grep /$playlist/, @IGNORE ) { next; }

# Turn the playlist name into a filename (this matches what Twonky does)
my $filename = $playlist;
$filename =~ s/\W/_/g;
$filename = 'iTunes-playlist-' . $filename . '##M.wpl';

# Open a new .wpl file
open (WPLFILE, ">$TWONKY_WPL/$filename") or warn("Could not open .wpl file for write: $!\n");

# Write file header
my $songs = @{ $playlists{$playlist} };
print WPLFILE "<?wpl version=\"1.0\"?>\n<smil>\n\t<head>\n\t\t";
print WPLFILE "<meta name=\"Generator\" content=\"TwonkyMedia\"/>\n\t\t";
print WPLFILE "<meta name=\"ItemCount\" content=\"$songs\"/>\n\t\t";
print WPLFILE "<meta name=\"Type\" content=\"M\"/>\n\t\t<meta name=\"Source\" content=\"iTunes\"/>\n\t\t";
print WPLFILE "<title>$playlist</title>\n\t</head>\n\t<body>\n\t\t<seq>\n";

# Write the track listings for this playlist
for ( my $i = 0; $i < $songs; $i++ ) {
# iTunes url encodes the file paths so we need to decode it first
$tracks{ $playlists{$playlist}[$i] } =~ s/\%([A-Fa-f0-9]{2})/pack('C', hex($1))/seg;
print WPLFILE "\t\t\t<media src=\"" . $tracks{ $playlists{$playlist}[$i] } . "\"/>\n";
}

# Write file footer
print WPLFILE "\t\t</seq>\n\t</body>\n</smil>";

close (WPLFILE);

# Set file permissions
`chown $FILE_OWNER:$FILE_GROUP "$TWONKY_WPL/$filename"`;
chmod( $FILE_PERMS, "$TWONKY_WPL/$filename" ) or warn("Could not set file permissions: $!\n");
}

exit 1;

END CODE ############################

For now this will only run when you restart the server. I was hoping that by simply replacing the twonky script with my own it would execute my script when it does it's updates. Unfortunately, it won't do that :( The code is a brute force solution although it does run reasonably quick. Sorry, about the lack of indentation. The forum strips all tabs and spaces but the code will still work. Post any questions or problems here.

Matt

Wires
Posts:44
Joined:Mon Jan 25, 2010 2:16 am
AV Hardware:PS3

Re: 6.01 Not Parsing iTunes Playlists Correctly

Post by Wires » Sun Dec 12, 2010 10:16 pm

It seems upgrading to 6.0.30 has killed my itunes playlists again. I even tried my tip above (deleting all things Twonky and rebooting). But nada. Tried rolling back to 6.0.28, and same crap.

I have playlists, lists the song quantity, the wpl's are listing fine, but no songs in the the playlists via Twonky.

Arggg... annoying....

Locked