finding non ASCII chars (that for example make Traktor unhappy)
Lukas
via web
please enter your message to us here
see here:
http://www.native-instruments.com/forum/showthread.php?t=80545
it seems Traktor (and I am getting the feeling from googling other apps have trouble with certain non-ASCII characters. Since Traktor doesn't provide a log with the necessary details, I wonder if beaTunes could come to the rescue?
if you experienced a crash or other abnormal behavior, you might want to attach log files from /Users/lsmith/Library/Logs/beaTunes
- beaTunes: 2.1.11
- JVM: Java HotSpot(TM) 64-Bit Server VM
- Version: 16.3-b01-279
- Vendor: Apple Inc.
- OS: Mac OS X 10.6.4, x86_64
- Registered as: Lukas Smith
Support Staff 2 Posted by hendrik on 15 Aug, 2010 09:14 PM
Hey,
I read part of that thread and it reminds me of some problems I had to solve with beaTunes a while back.
The issue was that
iTunes Music Library.xmlcontained malformed UTF-8 characters, which caused the XML parser to say goodbye. This is actually an iTunes problem - it should never write 'illegal' XML files, but it does sometimes, when id3 tags contain malformed data.The workaround was to simply ignore malformed characters.
-hendrik
3 Posted by dmuzii on 11 Jan, 2012 07:51 PM
Just the article I was looking for.
Totally get it that this is an iTunes problem. Does beaTunes have a tool in the application that could show songs with illegal characters so we can manually update and correct? Is there a list of potential illegal characters?
BeaTunes doe allow you to search on them.
Thanks in advance!
David!
Support Staff 4 Posted by hendrik on 13 Jan, 2012 11:07 AM
Hi David,
illegal and non-ASCII are actually two different things.
iTunes Music Library.xmlsays its encoded in UTF-8 and the characters I started ignoring simply weren't encoded in valid UTF-8. ASCII is not only an encoding like UTF-8, but both a character set and an encoding. Note that you can encode ASCII characters in UTF-8, but you can also encode Unicode characters in UTF-8. UTF-8 itself does not have characters, as it is not a character set, only a special way to write and read characters codes, which in turn are defined in character-sets. All a bit confusing, I know.Anyhow, to solve the issue, my question would be, what's the actual Traktor problem? Can it not deal with Umlauts, like ö (which is not an ASCII character).
Does it have trouble with characters not in Latin 1 (a.k.a. ISO 8859-1)? Or does it indeed have trouble with malformed UTF-8 characters?
That all said - it would be fairly easy to write a plugin that searches and replaces certain characters... But first you would need to know, what exactly to look for.
-hendrik
5 Posted by dmuzii on 13 Jan, 2012 04:27 PM
Hi Hendrik!
Thanks for replying back.
I was fairly generic in my previous email. Apologies for that.
I am using the Vitrual DJ (basic) program. Since my last thread, I have been able to pinpoint my issue in iTunes. Specifically, for my user account (not newly created user accounts), iTunes is unable to create/update the iTunes Music Library.xml file. If I create a new account on my MacBook Pro, then the this file is created/updated. And you know there's a problem with the software when you add a song to a newly created iTunes database and the iTunes Music Library.xml file is not created.
A senior level Apple Customer Service Representative captured the issue on my computer and is forwarding the details to their Software engineers. Let's see what they find out. I'll post their findings in about a week.
So, the initial problem in Virtual DJ was that I was not seeing all of my songs from playlists and smart playlists set up in iTunes. This is because the iTunes Music Library.xml file was not being updated. Initially, the Apple Customer Service Reps had suggested that the non UTF-8 characters were stopping the file from being created/updated. They suggested cleaning my 37,000 song library, which, at first, appeared to be a daunting task. Well I cleaned up the library and then disproved the non UTF-8 character issue as detailed in the previous paragraph.
I like your suggestion about writing a plugin to help identify non UTF-8 characters. I have reviewed your online notes but unfortunately I am not a programmer and am unable to follow along. Could you tell me where I could look for morne information on scripting?
Thank you again for all of your help.
best!
David!
Support Staff 6 Posted by hendrik on 18 Jan, 2012 10:02 AM
Hi David,
I'm sorry, but I think I have to disappoint you.
When reading
iTunes Music Library.xml, beaTunes skips all non-UTF 8 characters. Simply, because strictly speaking they aren't characters, just malformed data.So once you are in beaTunes, you can't see those bad boys anymore. They are simply not there.
If you were to replace one valid character with another - that would be possible (e.g. non-ASCII with
?). But that does not seem to be your problem.Cheers,
-hendrik