Bug:137

From Audacity Wiki
Revision as of 05:31, 25 June 2011 by Bibeksahu (talk | contribs) (Orphans on Sleep/Hibernate: Bibek's state, and comments about hooks. Apologies if this isn't the proper way to add to a bug report.)
Jump to: navigation, search

Summary Points


50 Comments.

  • "Orphaned" or "missing" blockfile errors are found on opening a project and when users accept the "recommended" course to delete, the project is silenced - possibly in a patchy way. The frequency of data loss is worrying/unacceptable for a stable release.
    • Sometimes this is associated with multiple projects open at the same time (or a project that has multiple "d*" folders inside "e00") and Audacity putting blockfiles in the wrong folder.
    • Sometimes this appears to be associated with edits at the end of a track (with a single project).
    • Audacity has occasionally been found to move or rename .au files within the _data folder when re-opening the project, thus creating orphans and/or missing files. This has never been reproducible even with copies of the original project where the problem happened.
    • There were some reproducible problems caused by UTF-8 characters and compression leading to missing blockfiles. These appear to be a (a) a separate issue, and (b) sorted now (by Michael).
  • Files belonging to the clipboard were incorrectly identified as orphans, but this was a separate 100% repeatable issue which is now fixed.
  • The problem used to be more reproducible by Gale, but has become less so with (the UTF-8??) fixes. Nevertheless it is still being reported frequently on 1.3.12.
  • One line of enquiry is related to orphans during hibernation. It may show we don't have code to recover from missed timer events.



Q&A

  • What is the approximate frequency of the bug reports now?
    • 10-12 per month (reported) with about half losing data.
    • As at 26May11 (6 weeks since 1.3.13 release) "only" 7 reports that appear to be to do with Bug 137. Some of these could be false clipboard orphans (the fix for these came after 1.3.13 Release), but some are definitely not, sounding like classic problems with having multiple projects open and lost data.
      • James That sounds too high a level to call it 'comet-returns'.
  • Is this associated with closing Audacity whilst Audacity is doing something - such as updating auto-save? (wild speculation from James)
    • No evidence of that. How long would autosave take on a 3 hour project? Doesn't the quit wait for autosave to finish?
  • Presumably detecting orphans does not lead to silencing, but detecting missing block files does?
    • Where Audacity has moved .au files between simultaneously open projects then orphans in one re-opened project will be missing audio in the other project. So deleting the orphans will permanently remove the ability to restore silenced audio in the other project.
  • Which fixes appear to have helped the problem?
    • The UTF-8 ones??
      • Gale: Passing .aup files between platforms is rarely mentioned in the bug reports. I have a hunch (unfounded, but it ties up chronologically) that the fixes to prevent unwanted zero length clips appearing may have helped.
  • Should orphan checking be at the subfolder level to catch more problems, rather than assuming as long there are no .au files not referenced in the .aup that the entire structure is correct? Missing file checking seems to be at the subfolder level, so theoretically Audacity should be able to check all folders in a project for a missing file and if found, put it in the correct subfolder indicated in the .aup.
    • Won't help files that are in the wrong project. Do we know that files are being moved into wrong sub folders?
      • Gale: Yes, several confirmed reports that happens, and I saw it happen myself as per the Summary.
        • James: That suggests the developers have made a nooby error, relying on current path when opening files rather than giving full path. If so, that would be very embarrassing, but easy to fix. It could also be a more subtle error with passing file handles between threads, which is a no-no because wxWidgets objects are typically not threadsafe.
  • What are the circumstances under which multiple "d*" folders are allocated inside the "e00" folder? I added to the release note that "having a long project with many tracks could cause the issue to occur" because I think this makes it more likely there will be multiple d* folders and this raises the risk of Audacity misplacing files. I've reviewed cases where people report cutting at the end of tracks as a problem but many are not quitting and restarting so I suspect they are probably seeing comment 23.
    • Projects with large numbers of blockfiles will have large numbers of d** folders under the e00 folder. Generally large numbers of blockfiles means large projects (high sampling rate, long duration, many tracks). It's also possible to temporarily have a large number of blockfiles by doing a lot of edits, since the history keeps them until you close the file.


Wording and Options in Dialog

Discussing in the wiki page, because it is a side issue to fixing the headline bug.

  • Gale: However if bug 137 is still in, remove "They should be deleted to avoid disk contention" in the orphan dialogue.
    • James: Agreed. We remove that text. Also 'delete files immediately' becomes 'delete files permanently'.


Orphans with Cut (comment 23)

A way to create 'orphans' but not the headline issue

  • Cutting audio places audio on the clipboard. After a save the clipboard audio files are copied to the new directory along with the audio that is actually part of the project. If you exit audacity, the clipboard is deleted and so are its audio files, which is correct. However if instead you close and re-open the saved file without exiting audacity the on-open-file checking sees these clipboard audio files and thinks they are orphans. Actually they are 'owned' by the clipboard. This is now fixed in r11118 with a follow-up fix in r11171.


Orphans on Sleep/Hibernate

  • (user to [email protected] 2011-April-7) "I'm using Audacity on a notebook with Win XP SSP 3. I normally do not shut down the notebook, but use the sleep funktion. After two or three times sleeping (while Audacity is opened) i can store Audacity .aup projects correctly, but if I want to open a project, Audacity finds a lot of "needless" files and proposes to delete them."
    • James Comments: This is fantastic, because it suggests a repeatable way to get Audacity to create orphans. It sounds as if this user's problem is coming because we update the dat files and only later update aup files, and we might be doing something 'wrong' with a timer. In this case timer is supposed to fire whilst we are hibernating, but of course doesn't, and audacity also does not see that missed timer event when it wakes up. Timer firing is not reliable, and we have no strategy for missed timer events (or something like). I would guess this user is triggering sleep whilst Audacity is actually doing something valuable. (We should also test with screen-savers).
    • Hmm, does he save the project before sleeping, or rely on auto-save?
      • Gale: I've removed user's name (unless he consents to have it published) because [email protected] is not under our privacy policy which allows publishing names. I'd love it if sleep or hibernate led us somewhere, though as I told him I have never found a problem so far with either sleep or hibernate.

        That said I am not following about "timers". I thought we had ripped that out. AFAIK every change to project state is saved in the autosave file when it happens (i.e. when the files in the _data folder change).

    • Bibek: I can confirm that I treat my laptop the same way (Vista SP2, 32-bit, 3G RAM), sleeping rather than shutting down, running Audacity 1.3.13 beta, and have seen the same problem. I haven't tracked it in as detailed a manner as the other user, but his description seems to fit my memory. I thought I had a set of files here somewhere that were botched up like this, but I can't find 'em now. If I at some point stumble across this again, I'll see if I can create a directly-reproducible test-case.
      • Also, I believe Windows supports some hooks related to sleeping / waking, so is it possible to add a sleep / wake hook in Audacity that just logs information about the state of the program to a file, in order to help debug the issue? The log may show an oddity even if the program appears to be running normally for the developers.

Folder deletion

Gale 26May11: See bug 409 for cases where empty subfolders are deleted. We should be sure this is not relevant to this bug (137).


Past Investigations

  • James pointed out in Code Review Triage that there is a FIX-ME in DirManager.cpp(843): "Might we get here without midkey having been set?" This function rebalances directory trees and needs very close scrutiny.
    • He's now had a look at it, and midkey always will be set. The rebalancing function still needs closer scrutiny, as when it 'fails' it fails silently, rather than reporting an error.

Bugzilla Extracts

We won't always paste extracts, due to redundant work involved.