AcousticBrainz Submit plugin crashes on very long tracks
I've encountered four instances now where the AcousticBrainz Submit analysis has crashed. The error message is:
"streaming_extractor_music.exe application error - unknown software exception (0x40000015) occurred in application at 0x00ab9b55"
The common element seems to be that the tracks being processed are all very long. In once case 41 minutes 46 seconds, in another 41:32, and in a third 38:36, and a fourth 1:03:32. It might instead be the actual files sizes rather than the track length. The shortest file is 88.3 MB. These are mp3 files encoded at 320kbps. I attached the shortest of the four files.
It is a repeatable error. I reanalyzed one of the tracks and it terminated with the same error.
- 04._Leiden_Mit_Manu.mp3 88.4 MB
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
1 Posted by Richard on 13 Jul, 2020 05:43 AM
Another track of length 35:08 seemed to process OK. "Distal Sonority" by M J Harris & Bill Laswell. I've been keeping my eye out for long tracks shorter than 38:36 to try to see where the cut-off for problems is.
Support Staff 2 Posted by hendrik on 13 Jul, 2020 07:17 AM
Thanks, for reporting this. I have contacted the AcousticBrainz people and let them know.
Support Staff 3 Posted by hendrik on 13 Jul, 2020 10:04 AM
Hi Richard,
I have been told, that the submission of tracks longer than 30 min is not super-useful for AcousticBrainz, especially when it's DJ sets that feature different parts.
I have therefore updated the plugin (new version 1.0.1) so that it skips very long tracks. Also, socket read timeouts have been increased and the error reporting (at least to the logs) has been slightly improved.
Please update via the plugins panel in the preferences.
Thanks!
-hendrik
4 Posted by Richard on 14 Jul, 2020 06:19 AM
Hendrik, How do I retrieve the 1.0.1 plug-in? I don't know where to download it from. Thanks.
Support Staff 5 Posted by hendrik on 14 Jul, 2020 07:18 AM
Please open the preferences, go to the "Plugins" tab, select the "Installed" plugins, right-click on the "AcousticBrainz Submit" plugin and select "Update plugin".
6 Posted by Richard on 14 Jul, 2020 07:24 AM
Hendrik, I did that. "Update plug-in" doesn't do anything. Literally, there is no response to me clicking "Update plugin". In fact, it might be grayed out. It's hard to tell.
Maybe this is another symptom of the problem I experienced of not being able to uninstall another plug-in? I tried to uninstall the 0.9.1 AcousticBrainz and the get Discogs Genre plug-ins, but they wouldn't go away.
Support Staff 7 Posted by hendrik on 14 Jul, 2020 11:16 AM
What version number do you see displayed for "AcousticBrainz Submit"?
Perhaps.
Can you please restart beaTunes, wait a couple of minutes, then go to the Plugin tab in preferences and try again?
If nothing happens, please upload your logs right afterwards.
Thanks!
8 Posted by Richard on 14 Jul, 2020 04:22 PM
I restarted and tried again. Maybe I hadn't restarted since you finished the 1.0.1 version.
This time, the Update worked in a sense. It did bring in the 1.0.1 version, but I was left with the 1.0.0 version being active and the 1.0.1 version being not loaded. I sent you logs at this point. "Before" and "after" screen caps attached.
Do I then need to uninstall the 1.0.0 version? I didn't try that yet.
9 Posted by Richard on 14 Jul, 2020 04:45 PM
Oh, now I see. I exited and restarted beaTunes again. Now in Preferences, 1.0.1 is active and 1.0.0 is crossed out. Screenshot attached.
I was tempted to go in and just delete items from the plugins folder to remove what I didn't want, but that seemed like a bad idea and not what you would expect a user to do.
I'll run 1.0.1 and see how it does. On my last batch with 1.0.0, it processed 28426 tracks. 491 of those had one of the 2 java errors I described, for a 1.73% failure rate. Most of those were of the "read timeout" variety. Of the 491, when those processed, 25 of those failed so I had to queue those 25 for a 3rd time. The 491 represented tracks that all did have a MBID, so the failure rates of the 28426 and the 491 shouldn't be compared. (Sorry, I just realized this continues a discussion started in another thread. The 1.0.1 update addresses separate problems covered in 2 distinct threads. That's my excuse!)
I'm running the analysis on all 4 cores of my CPU. I don't know if that has any bearing on the error rate. I wouldn't think so, but I thought I'd mention it. When I tried running just 1 thread, the error rate seemed comparable.
Support Staff 10 Posted by hendrik on 14 Jul, 2020 06:48 PM
That looks better. After the plugin update you should have seen a message box saying that you need to restart beaTunes for changes to take effect.
Actually that would have worked just fine!
Support Staff 11 Posted by hendrik on 14 Jul, 2020 06:49 PM
Please keep in mind that to a very large degree that AcousticBrainz plugin is just third party software. All the plugin does is making that piece of AcousticBrainz software more accessible and submitting the computation results to the project.
12 Posted by Richard on 14 Jul, 2020 07:50 PM
I didn't see a message box.
By the way, thanks so much again for all your support on this. I've been hammering (i.e., using) beaTunes pretty hard over the past couple of weeks. I report issues as I encounter them. As an atypical user, I know that problems I see or questions I may have, a lot of other users may never experience. Or, in some cases, won't care about.
13 Posted by Richard on 14 Jul, 2020 10:42 PM
I tested v1.0.1.
I analyzed a couple of very long tracks that had caused the plug-in to crash before. This time, they simply immediately vanished from the Task Queue with no error message. I assume this is the intended behavior, so everything is good there.
On the other errors, I have this to report:
I submitted a new batch of 2612 tracks to ABSubmit. Of those, there were 1345 cases where no MBID was found (which is not a concern at this time), 32 java "read timeout" errors, 6 java "file not found exception" errors, and 3 "method not allowed" errors I reported in another thread. The combined 39 java errors represent a 1.49% error rate. That is slightly better than the 1.73% rate reported from v1.0.0, but I don't know if it is a statistically significant difference. I located the 39 java errors in the message list by visual inspection, requeued them, and had to re-requeue 1 to complete the analysis of that batch.
Just as an additional thought that pertains to a different-but-related thread, it would be nice if the user (as a user option) could turn off the "MBID not found" error from appearing in the message list when that plugin is run. That would make finding and dealing with the remaining errors SO MUCH easier. It's a real chore locating 39 errors of interest in a list of 1387 error messages by scrolling and looking. Just throwing that thought out there.
Support Staff 14 Posted by hendrik on 15 Jul, 2020 07:30 AM
Yes, that's intended. The long tracks are fairly meaningless for AcousticBrainz. We simply ignore them.
What exactly are these?
Allowing to filter or search the message panel is an additional feature that will require some work. I like the idea, but want to prioritize fixing bugs over implementing new stuff.
15 Posted by Richard on 15 Jul, 2020 01:58 PM
An example of this error is "java.io.FileNotFoundException:" followed by a filename like "C:\Users\Richard\AppData\Local\Temp\copy6362950677360790880.mp3"
I described these in the thread titled "Options to manage Message Queue". Unfortunately, I didn't provide a screen shot of that in the thread. (Those particular errors weren't the direct subject of the thread.) I seem to be getting relatively more of this type now than I was in v1.0.0 compared to the java read timeout errors, although still more "read timeouts" in absolute terms. (However, in a small run today, out of 199 items, I had 1 Read Timeout and 3 FileNotFoundExceptions.) These errors are not repeatable - which is why I look for them in order to "fix" them. If I requeue tracks that received either of these 2 java errors, they will usually process OK the 2nd time around. (If one errors out the 2nd time, a 3rd queuing has always worked so far.)
I try to stay on topic in these different discussion threads, but sometimes it's difficult as the topics are interrelated.
I appreciate your thoughts on adding functionality to the message panel. I completely understand the need to prioritize.
16 Posted by Richard on 15 Jul, 2020 02:27 PM
Here is a fresh example of the FileNotFoundExceptin error.
I just did 1 album this morning. I actually deleted the album from beaTunes (which deleted the files, too) and replaced them with new mp3 files which I had edited in Audacity to get rid of excessive silence at the beginning of each track. When I added the tracks back in and then did the full analysis after a new resync, 4 of the first 5 tracks threw the FNFE error. Very unusual compared to ABSubmit v1.0.0, and kind of disheartening that my requeue rate is so high on just 11 tracks. (Screenshot attached)
Support Staff 17 Posted by hendrik on 16 Jul, 2020 06:16 AM
Hey Richard,
if you can reproduce the issue, can you please do so and upload a fresh set of logs right afterwards, so that I can take a look where exactly that exception come from?
Also: Can you play those songs with no problems in beaTunes?
Thank you!
-hendrik
18 Posted by Richard on 16 Jul, 2020 03:12 PM
I can play the songs, no problem.
The issue isn't repeatable with the songs that cause the exception. Every time I've requeued those tracks for the same analysis, they analyze OK. No error.
I've been running most of these analyses with 4 parallel tasks, the max available to me. I've got a lot of tracks to process and I'd like to get them done ASAP. I had wondered if resource contention might be a contributing factor. I had run for awhile with just 1 or 2 parallel tasks at different times. When i did that before, it didn't seem to reduce the incidence of exceptions.
But last night overnight, I set the parallel tasks back to 1 again. I know I have to restart restart beaTunes when I change that parameter, so I do. Out of about 3000 songs, there weren't any FNF exceptions, and there were only 5 Read Timeout exceptions. Based on what I was seeing before, I would have expected around 50. So maybe resource contention is a factor after all. I'll keep running at 1 task for awhile and see what happens. (Using just 1 parallel task, it will take about 17 days of continuously running ABSubmit to finish doing my 104,000 remaining unprocessed tracks. So you can see why I'm trying to speed things up wherever possible.)
If I can catch a FNF exception in real time, I will send the logs and advise you. Thanks!
Support Staff 19 Posted by hendrik on 16 Jul, 2020 04:06 PM
Just to explain a bit:
The read timeout is probably the AcousticBrainz server not being able to process all your data fast enough to send back a response before the read timeout expired. It is entirely possible that your submission still goes through and is stored, even though you see this error.
The FNFE is something on the beaTunes end, i.e., something I might be able to debug and fix once I know more about it.
Cheers!
20 Posted by Richard on 21 Jul, 2020 03:54 AM
Hendrik:
I just uploaded my logs. I finished analyzing a big batch of tracks, and then a small batch of 11. I resubmitted one of the big batch and then noticed one of the little batch failed with the FileNotFoundException error. The track is called "Time". After it failed, 9 more tracks ran through (one of which had the "Unable to find MBID" error"), and then I submitted one more track before realizing "Time" had the error. So hopefully the logs captured the analysis approx 10 songs prior to the most-recent one I did. I hope this shows you something.
I was running 3 parallel processes, so the order that I submitted the songs might not be exactly the order in which they processed.
I attached a screen shot of the error message.
Richard
PS- I resubmitted "Time". As expected, the error did not repeat.
Support Staff 21 Posted by hendrik on 22 Jul, 2020 04:49 PM
Hey Richard,
thanks for catching this and sending your logs!
I believe the FNFE issue is fixed in v1.0.3 of the plugin, which I've just released.
Cheers,
-hendrik
22 Posted by Richard on 22 Jul, 2020 07:01 PM
Thank you Hendrik.
Since my previous post in this thread, I ran a batch of about 5200 songs through ABSubmit. There were 2286 error messages, including 8 FNFE errors. (For completeness, the other errors were 57 Read Time Out, 1 Connect Time Out, and 2220 Unable to Find MBID. The "Connect Time Out" is a new one I'd only seen once or twice before EXCEPT for about an hour when my internet was down recently, and I saw a whole bunch of them I didn't know the internet was down, so the Task Queue was processing items with no internet for awhile.)
So that's the most recent baseline: about 0.35% of the errors were FNFE in that batch. I just updated to 1.0.3 and will resume the task queue with 7374 remaining ABSubmits to process. I will report tomorrow how it goes.
***I just now remembered there is documentation that says do NOT update a plug-in with items in the task queue. I'd forgotten about that. I hope it won't be a problem in this case! I did do a Pause Analysis, Restart beaTunes, update the plugin, Restart beaTunes again, and then Resume Analysis.
Support Staff 23 Posted by hendrik on 23 Jul, 2020 08:34 AM
I don't think that for ABSubmit this is an issue, if you're just updating.
Regarding timeouts, those are more or less out of my hands. If the service we are talking to does not answer within a certain time, that's what happens.
24 Posted by Richard on 23 Jul, 2020 04:11 PM
I am happy to report that there were NO FNFE errors in the batch of 7374 I submitted under 1.0.3. It looks like you nailed it!
I still have roughly another 55,000 still to process that I haven't submitted yet, split over several batches. If anything new pops up, I will report it. Otherwise, no news will be good news.
Support Staff 25 Posted by hendrik on 24 Jul, 2020 07:08 AM
Nice! :-)
Thank you for reporting and testing.
Richard closed this discussion on 06 Aug, 2020 05:33 PM.