Abysmal performance on replaygain analysis

iandphillip's Avatar

iandphillip

23 Mar, 2020 04:47 PM

Hi. I bought this app to use solely as a replaygain analyser for my music library. I used to use Audirvana for playback, and it had a rather basic but usable analyser built in. My needs have changed though and I now need a standalone solution.

The library is a little over 13,000 tracks, and Audirvana got through them in several minutes. Trying to use beaTunes though, it's taken an hour to get through just the first few tracks, and says it'll take six days(!) to finish. Everything in the analyser dialogue is turned off except replaygain, so I can't understand why its so unbelievably slow. Am I missing something, or is this app just designed to be used with a handful of tracks at a time?

  1. Support Staff 1 Posted by hendrik on 23 Mar, 2020 04:54 PM

    hendrik's Avatar

    Hi,

    What ReplayGain settings did you use?
    Track and/or album?
    What kind of peak detection?

    Thanks,

    -hendrik

  2. 2 Posted by iandphillip on 23 Mar, 2020 05:08 PM

    iandphillip's Avatar

    Hi, thanks for the fast reply. It's set to do both track and album with sample peak. I had it on EBU R128 at -23db.

  3. Support Staff 3 Posted by hendrik on 23 Mar, 2020 05:26 PM

    hendrik's Avatar

    Hm..

    it's taken an hour to get through just the first few tracks, and says it'll take six days(!) to finish

    The ETA beaTunes is giving you may be inaccurate, especially when calculating album replay gain, because it is actually not a per-track task.

    But it should definitely not take an hour for the first few (I assume less than 10) tracks. That is waaaay too long. Are the files locally stored or on some kind of network drive? Can you please upload your logs via Help -> Upload Logs. I'd like to take a closer look.

    Thanks,

    -hendrik

  4. Support Staff 4 Posted by hendrik on 23 Mar, 2020 05:27 PM

    hendrik's Avatar

    And in any case, if you're unhappy, I can of course offer a refund.

  5. 5 Posted by iandphillip on 23 Mar, 2020 05:55 PM

    iandphillip's Avatar

    Hi Hendrik. The files are on a Synology NAS. I'm trying chunking the process into several albums at a time, rather than hitting the entire library at once. That's certainly progressing better.

  6. 6 Posted by iandphillip on 23 Mar, 2020 05:58 PM

    iandphillip's Avatar

    Logs uploaded now. Thanks.

  7. 7 Posted by iandphillip on 23 Mar, 2020 07:49 PM

    iandphillip's Avatar

    Yeah, so it's just hanging again…

  8. Support Staff 8 Posted by hendrik on 24 Mar, 2020 02:22 PM

    hendrik's Avatar

    Hey Ian,

    quick note to let you know, I have received your logs and am working in this.

    Thank you for your patience,

    -hendrik

  9. 9 Posted by iandphillip on 24 Mar, 2020 02:45 PM

    iandphillip's Avatar

    Thanks Hendrik. I’m currently in the process of copying my library across to an external drive to see if USB might be more successful than trying to do this across a network. Sadly my internal SSD is just way too small to store the files locally.

  10. 10 Posted by iandphillip on 24 Mar, 2020 03:13 PM

    iandphillip's Avatar

    Ok, so that's nailed it. Totally different performance. It's still hung on one track, but I'm putting that down to a likely damaged file. I guess I was just being unrealistic about what to expect across a network. Thanks again.

  11. Support Staff 11 Posted by hendrik on 24 Mar, 2020 03:16 PM

    hendrik's Avatar

    Yes and no. I believe beaTunes makes too many network calls, which may slow things down additionally. Working on that right now.

  12. Support Staff 12 Posted by hendrik on 25 Mar, 2020 11:11 AM

    hendrik's Avatar

    Hey Ian,

    I've made some changes to how data is loaded and also fine tuned some of the decoding and filtering necessary for ReplayGain/EBU.

    Could you please download and install https://www.beatunes.com/download/beaTunes-5-2-7.dmg ?

    This release is not yet public, but contains all the changes and is of production quality. In my tests, local ReplayGain computation was about 50% faster (measured with an SSD). But I wasn't able to test this on a NAS or on an external HDD. Any chance you could run some analysis on your Synology setup and then upload fresh logs, so that I can see the analysis times?

    Just some additional info: Many of your songs are AIFFs, which are very large. Obviously it takes much longer to send those over the network than for example MP3s. This made the issue extra abysmal.

    Regarding "hanging": When doing Album ReplayGain analysis, beaTunes will try to find all songs of an album when encountering the first song of an album. At that point, it may look as if beaTunes is hanging, when it's actually processing all those other tracks belonging to the same album. Just leave it running...

    Thanks for pointing me to this,

    -hendrik

  13. 13 Posted by iandphillip on 25 Mar, 2020 09:57 PM

    iandphillip's Avatar

    Hi Hendrik

    I've downloaded 5.2.7 and run a couple of albums through it across the network. Whatever you've done, I can confirm it does appear significantly quicker. I've uploaded the logs as requested.

    I got my full library analysed yesterday using a USB3 HD and the process went fairly quickly and entirely smoothly. I'm sure an SSD would've been much faster, but the spinning disc did the job fine.

    I need to thank you for your help with this. I feel bad about 'abysmal' — I should explain that I wasn't feeling at all well at the time, and shouldn't have been playing around with my music library at all. I think the frustration just all got the better of me. I offer that as an explanation, not an excuse. Sick or not, it's still a dick term to use.

    If you're happy that you've got to the bottom of the issue, and there's nothing more you need from my end, please feel free to delete this thread. No developer needs a post with 'abysmal' in the heading littering up their support system.

    Thanks again.

    — Ian

  14. Support Staff 14 Posted by hendrik on 26 Mar, 2020 07:30 AM

    hendrik's Avatar

    Hi Ian,

    I've downloaded 5.2.7 and run a couple of albums through it across the network. Whatever you've done, I can confirm it does appear significantly quicker. I've uploaded the logs as requested.

    I hate to contradict, but according to the logs it does not seem faster at all, which makes me think that the bottleneck is indeed the network, not beaTunes.

    I got my full library analysed yesterday using a USB3 HD and the process went fairly quickly and entirely smoothly. I'm sure an SSD would've been much faster, but the spinning disc did the job fine.

    I guess this supports the network-bottleneck theory.

    I've noticed that you are using AFP as protocol. Synology NASs usually also support SMB—perhaps that has better performance (there are plenty of discussion of SMB vs AFP on the Synology board, e.g. here). You could also try to reduce the number of parallel tasks (currently 8, I believe) to reduce network congestion.

    I need to thank you for your help with this. I feel bad about 'abysmal' — I should explain that I wasn't feeling at all well at the time, and shouldn't have been playing around with my music library at all. I think the frustration just all got the better of me. I offer that as an explanation, not an excuse. Sick or not, it's still a dick term to use.

    If you're happy that you've got to the bottom of the issue, and there's nothing more you need from my end, please feel free to delete this thread. No developer needs a post with 'abysmal' in the heading littering up their support system.

    Software that does not do what you want it to do is a source of great frustration. You are certainly not alone with the need to vent every now and then. I have read much worse. That said, I really appreciate your explanation, especially as it wasn't strictly necessary. Users, developers, and those providing support—we all deserve to be treated with respect. Explaining your emotional situation does show me that respect. Thank you.

    Unless you insist, I'm not going to delete this discussion. Other people may have the same problem and can benefit from the information.

    Cheers,

    -hendrik

  15. 15 Posted by iandphillip on 26 Mar, 2020 08:10 AM

    iandphillip's Avatar

    Unless you insist, I'm not going to delete this discussion. Other people may have the same problem and can benefit from the information.

    Entirely your call.

    Thanks again.

  16. Support Staff 16 Posted by hendrik on 26 Mar, 2020 10:13 AM

    hendrik's Avatar

    Should you ever try out SMB instead of AFP, please let me know, how that goes.
    For the more general issue, I have posted a StackOverflow question. Perhaps somebody knows the answer to this...

  17. 17 Posted by iandphillip on 26 Mar, 2020 02:32 PM

    iandphillip's Avatar

    Hi. I turned off AFP and tried connecting over SMB. It might be a little quicker (although we've established I'm not a great judge of that). I've run a few passes over tracks in a folder on the same NAS and sent the logs if that helps.

  18. Support Staff 18 Posted by hendrik on 26 Mar, 2020 03:31 PM

    hendrik's Avatar

    Thanks for doing this!

    The logs says, SMB is about 10x faster.

    Here are some analysis times for AFP:

    2020-03-25 21:45:08,345 [182016] [DEBUG][AnalysisQueueWorker-2] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 47613ms
    2020-03-25 21:45:08,459 [182130] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:08,629 [182300] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:14,905 [188576] [DEBUG][AnalysisQueueWorker-1] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 42550ms
    2020-03-25 21:45:15,165 [188836] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:15,334 [189005] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:22,272 [195943] [DEBUG][AnalysisQueueWorker-7] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 62970ms
    2020-03-25 21:45:22,386 [196057] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:22,541 [196212] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:47,556 [221227] [DEBUG][AnalysisQueueWorker-2] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 38872ms
    2020-03-25 21:45:47,701 [221372] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:47,830 [221501] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:50,906 [224577] [DEBUG][AnalysisQueueWorker-5] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 49479ms
    2020-03-25 21:45:51,005 [224676] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:51,153 [224824] [DEBUG][AudioMetaData] com.tagtraum.core.FileId: Unsupported path store type: afpfs
    2020-03-25 21:45:51,344 [225015] [DEBUG][AnalysisQueueWorker-4] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 49145ms
    

    Here are some for SMB:

    2020-03-26 14:28:46,850 [ 91720] [DEBUG][AnalysisQueueWorker-1] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 2570ms
    2020-03-26 14:28:48,010 [ 92880] [DEBUG][AnalysisQueueWorker-2] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 3743ms
    2020-03-26 14:28:48,168 [ 93038] [DEBUG][AnalysisQueueWorker-0] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 3899ms
    2020-03-26 14:28:48,560 [ 93430] [DEBUG][AnalysisQueueWorker-3] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 4286ms
    2020-03-26 14:28:50,455 [ 95325] [DEBUG][AnalysisQueueWorker-0] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 2275ms
    2020-03-26 14:28:50,859 [ 95729] [DEBUG][AnalysisQueueWorker-3] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 2233ms
    2020-03-26 14:28:51,246 [ 96116] [DEBUG][AnalysisQueueWorker-2] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 3201ms
    2020-03-26 14:28:52,181 [ 97051] [DEBUG][AnalysisQueueWorker-3] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 1297ms
    2020-03-26 14:28:53,044 [ 97914] [DEBUG][AnalysisQueueWorker-2] com.tagtraum.beatunes.analysis.AudioPumpAnalysisTask: Pump took 1515ms
    

    So unless we are seeing some caching effects, SMB wins this hands down.

  19. 19 Posted by iandphillip on 26 Mar, 2020 09:02 PM

    iandphillip's Avatar

    Well, I'd say that identifies the bottleneck. I'll keep AFP well and truly off from now on.

  20. hendrik closed this discussion on 29 Mar, 2020 09:45 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac