beaTunes v5 EA1 crash - OutOfMemoryError while analyzing
Hi,
When analysing my library with beaTunes v5 EA1 it crashes. Looking at the logs I can see the following error:
2017-05-30 12:41:36,261 [3144267] [ERROR][AnalysisQueueWorker-1] com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker: OutOfMemoryError while analyzing. java.lang.OutOfMemoryError: Java heap space at com.tagtraum.jipes.audio.AudioBufferFunctions$AudioBufferMapFunction.map(AudioBufferFunctions.java:241) at com.tagtraum.jipes.audio.AudioBufferFunctions$AudioBufferMapFunction.map(AudioBufferFunctions.java:229) at com.tagtraum.jipes.universal.Mapping.processNext(Mapping.java:62) at com.tagtraum.jipes.AbstractSignalProcessor.process(AbstractSignalProcessor.java:123) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:144) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:117) at com.tagtraum.jipes.AbstractSignalProcessor.process(AbstractSignalProcessor.java:124) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:144) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:129) at com.tagtraum.jipes.audio.InterleavedChannelSplit.process(InterleavedChannelSplit.java:76) at com.tagtraum.jipes.audio.InterleavedChannelSplit.process(InterleavedChannelSplit.java:28) at com.tagtraum.jipes.SignalPump.process(SignalPump.java:176) at com.tagtraum.jipes.SignalPump.pump(SignalPump.java:143) at com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask.a(SourceFile:273) at com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask.runBefore(SourceFile:148) at com.tagtraum.beatunes.analysis.Task.run(SourceFile:745) at com.tagtraum.beatunes.analysis.Task.run(SourceFile:753) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker.a(SourceFile:150) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker$$Lambda$771/299453965.call(Unknown Source) at com.tagtraum.beatunes.library.AbstractMusicLibrary.serialExecute(SourceFile:242) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.tagtraum.core.wrap.WrappingProxyFactory$TunesLibraryInvocationHandler.a(SourceFile:66) at com.tagtraum.core.wrap.WrappingProxyFactory$TunesLibraryInvocationHandler.invoke(SourceFile:59) at com.sun.proxy.$Proxy4.serialExecute(Unknown Source) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker.run(SourceFile:1125) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-05-30 12:41:36,261 [3144267] [ERROR][AnalysisQueueWorker-1] com.tagtraum.beatunes.BeaTunes: OutOfMemoryError java.lang.OutOfMemoryError: Java heap space at com.tagtraum.jipes.audio.AudioBufferFunctions$AudioBufferMapFunction.map(AudioBufferFunctions.java:241) at com.tagtraum.jipes.audio.AudioBufferFunctions$AudioBufferMapFunction.map(AudioBufferFunctions.java:229) at com.tagtraum.jipes.universal.Mapping.processNext(Mapping.java:62) at com.tagtraum.jipes.AbstractSignalProcessor.process(AbstractSignalProcessor.java:123) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:144) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:117) at com.tagtraum.jipes.AbstractSignalProcessor.process(AbstractSignalProcessor.java:124) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:144) at com.tagtraum.jipes.SignalProcessorSupport.process(SignalProcessorSupport.java:129) at com.tagtraum.jipes.audio.InterleavedChannelSplit.process(InterleavedChannelSplit.java:76) at com.tagtraum.jipes.audio.InterleavedChannelSplit.process(InterleavedChannelSplit.java:28) at com.tagtraum.jipes.SignalPump.process(SignalPump.java:176) at com.tagtraum.jipes.SignalPump.pump(SignalPump.java:143) at com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask.a(SourceFile:273) at com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask.runBefore(SourceFile:148) at com.tagtraum.beatunes.analysis.Task.run(SourceFile:745) at com.tagtraum.beatunes.analysis.Task.run(SourceFile:753) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker.a(SourceFile:150) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker$$Lambda$771/299453965.call(Unknown Source) at com.tagtraum.beatunes.library.AbstractMusicLibrary.serialExecute(SourceFile:242) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.tagtraum.core.wrap.WrappingProxyFactory$TunesLibraryInvocationHandler.a(SourceFile:66) at com.tagtraum.core.wrap.WrappingProxyFactory$TunesLibraryInvocationHandler.invoke(SourceFile:59) at com.sun.proxy.$Proxy4.serialExecute(Unknown Source) at com.tagtraum.beatunes.analysisqueue.AnalysisTaskWorker.run(SourceFile:1125) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
Running this on a Windows 10 Creators Update. Intel Core i7-4770K w/ 16GB RAM.
Let me know if you need any more info.
Thanks,
Olly
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
Support Staff 1 Posted by hendrik on 30 May, 2017 12:06 PM
Hey Olly,
thanks for testing. Please restart beaTunes and upload your logs via the
Helpmenu.Thanks!
-hendrik
2 Posted by Olly Ainger on 30 May, 2017 12:17 PM
Sure, I’m just running another analyse now so if/when it crashes again I’ll upload the logs.
I notice that my max Memory allocation is 512m and I’ve got most of the analysis options turned on and your knowledge base suggests I should try turning some off if I get this error.
Can I tweak the BEATUNES_MEMORY=-Xmx512m environment variable and give beaTunes 1024m instead, would that help?
Support Staff 3 Posted by hendrik on 30 May, 2017 12:25 PM
Yes, it would definitely help.
Support Staff 4 Posted by hendrik on 30 May, 2017 12:26 PM
Tip: You can monitor memory consumption in beaTunes About dialog.
Support Staff 5 Posted by hendrik on 30 May, 2017 12:30 PM
To give me some context:
How many songs are in your library?
6 Posted by Olly Ainger on 30 May, 2017 01:18 PM
I have 3403 songs in my library currently.
The analysis just crashed again so I have uploaded the logs for you.
This time however, I received a message to increase my max heap size to 1GB which I've now done. I'll run the analysis again and let you know how I get on.
Many thanks.
Support Staff 7 Posted by hendrik on 30 May, 2017 01:44 PM
Thanks for the logs.
Increasing the memory to 1GB should do the trick. But I guess I have to check, what's using so much memory... Album ReplayGain is a prime candidate.
-hendrik
8 Posted by Olly Ainger on 30 May, 2017 01:53 PM
That might make sense, I have quite a few compilation albums that can span many, many tracks. Is it trying to work out the Album ReplayGain for all the songs at once for a given compilation perhaps!?
Support Staff 9 Posted by hendrik on 30 May, 2017 02:48 PM
Hm. Looks more like a problem with the new key detection...
10 Posted by Olly Ainger on 30 May, 2017 02:52 PM
☹
Let me know if there’s anything else you need from me to help.
Current analyse is still running about 20% through which is the furthest it has got. I did notice that the heap utilisation peaked at around 90% of the now 1GB allocation so it’s cutting it close. I’ll let you know if any more problems.
Support Staff 11 Posted by hendrik on 30 May, 2017 03:11 PM
I think I'll have this fixed in the next 45 min.
I can then send you a link for an updated version.
12 Posted by Olly Ainger on 30 May, 2017 03:23 PM
Nice work!
Happy to test back at 512MB and re-analyse from scratch for you with the new version, I’m in no rush for the results
Support Staff 13 Posted by hendrik on 30 May, 2017 03:55 PM
Hey Olly,
here are the promised links:
Please first shut down beaTunes, then download and install.
Hopefully, this will fix the memory issue. In any case, please let me know!
Thank you!
-hendrik
14 Posted by Olly Ainger on 30 May, 2017 04:03 PM
Brilliant! Trying it out now.
I've sized the memory back down to 512MB and I'll keep you posted.
Thanks.
Olly
Support Staff 15 Posted by hendrik on 30 May, 2017 04:04 PM
Awesome!
Support Staff 16 Posted by hendrik on 31 May, 2017 07:09 AM
Good morning Olly,
did the EA2-Snapshot solve the memory issue you reported?
Thanks,
-hendrik
17 Posted by Olly Ainger on 31 May, 2017 07:16 AM
Crashed again I’m afraid with out of memory... Sending logs for you now.
Support Staff 18 Posted by hendrik on 31 May, 2017 07:18 AM
:-(
Did it run for longer this time?
19 Posted by Olly Ainger on 31 May, 2017 07:22 AM
Yes, it ran for much longer, crashed about 50% through the library this time.
Unfortunately the send logs is not working from the Help menu so please find the logs folder zipped and attached here.
Support Staff 20 Posted by hendrik on 31 May, 2017 08:18 AM
Thanks for the logs.
How many songs are in
Anthems: Alternative 80s - Ministry of Sound?-hendrik
PS: Feel free to delete those
gc-14...logfiles in your log directory. Once they are gone, the regular upload will work again.21 Posted by Olly Ainger on 31 May, 2017 09:30 AM
Hi, there are 59 songs in that compilation!
Support Staff 22 Posted by hendrik on 31 May, 2017 09:41 AM
Unfortunately, I'm having trouble reproducing the issue.
Can you please shut down beaTunes and then download and install https://www.beatunes.com/download/beaTunes-5-0-0-EA2-HEAPDUMP-SNAPS... ?
It does not contain a fix, but dumps its current heap (memory) to disk, if an
OutOfMemoryErroroccurs. This heap dump should allow me to figure out what's taking up all this memory. Unfortunately, this procedure has a downside: the heap dump is a huge file. Any chance you can send me that file should a memory problem occur again? This only makes sense with a memory heap of 512M—any larger and the dump really gets too big.You can find the heap dump in
C:\Users\oaing\AppData\Local\Temp\(your%TMP%directory). The file itself is namedbeaTunes-heapdump.hprof. You should be able to send it using a service like https://wetransfer.com.Thanks so much for your help!
-hendrik
Support Staff 23 Posted by hendrik on 31 May, 2017 09:46 AM
I forgot to mention: please send the transfer notification to [email blocked].
Thanks.
24 Posted by Olly Ainger on 31 May, 2017 10:32 AM
FYI, I installed this version over the top of the last but beaTunes
wouldn't launch.
I've uninstalled and re-installed now which fixed that, set heap back to
512m and re-running the analysis from scratch. It may take a while to get
to the same point again...
I'll let you know.
Support Staff 25 Posted by hendrik on 31 May, 2017 12:01 PM
Did you try to install while beaTunes was running?
That's what makes memory issue so nasty to debug... :-(
I'm still running my own tests here—if I can come up with something, I'll also let you know.
Thanks!
Support Staff 26 Posted by hendrik on 31 May, 2017 03:02 PM
I think I may have found something else. Will post a new snapshot later on.
27 Posted by Olly Ainger on 31 May, 2017 03:27 PM
It's just crashed, find the dump here:
https://www.dropbox.com/s/s1qzregxxfedodr/beaTunes-heapdump.7z?dl=0
Support Staff 28 Posted by hendrik on 31 May, 2017 03:58 PM
Got it! Thanks!
It looks like there are a ton of writes to iTunes that are queued. Any chance your iTunes has a dialog open? Or is otherwise misbehaving?
29 Posted by Olly Ainger on 31 May, 2017 04:18 PM
It was misbehaving, but I thought that was due to beaTunes...
About 30mins before the crash I tried to look at iTunes, it was minimised
so tried expanding it. It would not maximise so it appeared to have hung.
After beaTunes had crashed, iTunes was still unresponsive and had to be
task killed.
iTunes works OK whilst analysing so unclear at what point it becomes
unresponsive. I've attached SysInternals procmon to iTunes.exe and running
another analyse to see if I can see what could be causing the application
to hang, but this will be a bit noisy. Maybe some defensive code is needed
around X queued writes so at least beaTunes doesn't crash?
Support Staff 30 Posted by hendrik on 31 May, 2017 04:51 PM
I'm building a new snapshot that avoids queuing too many tasks. However, for now this will have only the effect that there'll be error messages all over the log, probably not in the UI (don't know, haven't tested yet). This means beaTunes probably won't crash with OOME anymore, but it really won't work either if it cannot communicate with iTunes. Cause that's what really happened here.
I will do some more testing on Windows 10 tomorrow.