Talk:Release Checklist

Matters to be tidied/ integrated after closing Checklist

 * Use Bugzilla for lobbying for P number changes.
 * (March 2010) Experimentally, use the "Patches" product for new patches submitted by people without "bug wrangler" status. Attach the patch if necessary to an actual bug in the "Audacity" product, while retaining the issue in the "Patches" product for ease of reference. This idea sees "Patches" as not a bug list but a list of potential patches for potential issues. Try keeping the "Patches" product out of bug searches; instead, make a routine to look at the Patches product separately as a "pending in-tray".
 * Occasional use of Wording to record formulated policy, like being consistent about plug-in.
 * Normally when we make users bug-wranglers we will not give them the rights to make-or-unmake other people bug-wranglers. We did for our early bug-wranglers, and if that ever causes a problem we (admins) will change that.

Gale's views

 * 1) Old Bugzilla (+1 for legibility!) that I'll cull for anything there still useful.
 * 2) * Update: Matt Brubeck has withdrawn that but I've asked him for the data which he has still kept
 * 3) Old Bugzilla has an attached audacity-bugs SF list that we need to decide what to do with.
 * 4) A list of old bugs and developer-/QA-supported enhancements at Release checklist not aiming for 1.4. I suggest this content should slowly be moved into Bugzilla or developed into a new Proposal.
 * 5) * JC: +1
 * 6) Release Checklist Watching Brief was supposed to track what we now call Moonphase and Heisenbugs, but was largely ignored. We now have several Moonphase bugs where there is debate over them being P2 or P3. GA thinks they are P2 (for a minority of those affected, they make 1.3 near unusable). I agree with having a new page Moonphase and Heisenbugs to track how many reports we receive of these problems, and collect detailed symptoms. This may help us make a more rational decision on their importance vis-a-vis what our quality aims for non-Beta are.
 * 7) * JC: +1
 * 8) Wiki Development Checklist (for both this Wiki and the Manual Wiki) and Forum Development Checklist
 * 9) * Relevant items include security issues, bug fixes and enhanced functionality through installing additional extensions. I think these can stay as is.
 * 10) * JC: +1. Perhaps a better name needed, but name-change not vital.
 * 11) I suggest a new Bugzilla Development Checklist to track issues with the bug tracker. This includes both things we can change and bugs that are down to the software. Noted so far:
 * 12) * Inadequate contrast especially on bug-editing pages
 * 13) * Overwide Release Notes box on bug-editing pages
 * 14) * If you make field changes such as changing Priority and editing Steps to Reproduce, then attach a file, the changes before attaching are discarded
 * 15) * No way to add release notes when creating a bug
 * 16) * Adding attachments is ignored in history
 * 17) * Replying to a comment only links to the comment being replied to, which could be several scrolls up; it should cite the text
 * 18) * "Default Asignee" in bug-editing pages
 * 19) ** JC: Use wiki for bugzilla enhancement requests.
 * 20) ** I've removed manual as a product and added patches. To edit audacity bugs you need to be a 'Bug Wrangler' (everyone who has an account right now is).
 * 21) *** Gale: Thanks, James. OK I tested it by logging in as another user and changing their rights. If user is not a member of any groups, they can create a new issue in Patches but can't view any bugs outside of Patches. Adding them to "canconfirm" makes no difference. Adding them to the "editbugs" group makes no difference (previously that group let you add a new (Audacity) bug). It follows that no-one logged out can now view bugs in the Audacity product, either. I think we don't want to stop anyone *looking* at the bug list, especially not a logged in user who may want to look for a bug to develop a patch for - they just should not be able to edit the Audacity bug in any way, or add a new one.
 * 22) *** JC: Ah. I've changed things and now anyone can see audacity bugs, logged in or not, but only Bug Wranglers can edit them.
 * 23) *** Gale: Also I note that Bug Wranglers can make anyone else a Bug Wrangler (or remove them from that group). Shouldn't that right be confined to an admin?
 * 24) *** JC: I believe that's down to a choice I made when I added bug-wrangler to each existing user and not a property of the group. I could have not given people the right to make others bug-wranglers.  I felt it's fine for our existing users.  I'm fine with a different policy too.
 * 25) *** Gale: I think it probably is fine for the existing users, but there may be a slight risk later on if people we don't really know become bug wranglers. I'd have a mild preference that only admins. can modify a user's Bug Wrangler status.
 * 26) *** JC: OK. When either of us makes someone new a bug wranglers then we just don't tick the 'can modify' flag for them.
 * 27) ** As a workaround I've added links for new bug and new patch to the message of the day.
 * 28) ** We might be able to open up Bugzilla to general editing, since I believe I got it right and only Bug Wranglers can edit audacity bugs. No hurry there.
 * 29) *** Gale: Making Bugzilla available to anyone to create an account, log in and add a "patch to be triaged" is probably OK, but only IMO if we restrict "advertising" Bugzilla to SubmittingPatches. Being pessimistic, I could easily see a user finding Bug Lists, going to Bugzilla, logging in and creating an issue that Audacity can't record stereo mix, or asking how to make his USB turntable record in stereo. I think something with "buttons to press" may be more appealing than a Wiki. I think at the moment I'd rather make Bugzilla more visible (to anyone) but require vetting. Users can either say why they want access to edit/create Audacity bugs, or they can e-mail their patch to me (or whoever) to be given access only to Audacity Patches. That should be little more delay to them than signing up and confirming. Plus, are we secure against bots logging in and doing things? Clearly bots do manage to register themselves on the Forum even with the confirmation routine there (32 spam posts in about three minutes).
 * 30) *** JC: +1. If I have it right, they'll need an account AND bug-wrangler to create and edit audacity bugs but an account will be enough for the patches product.
 * 31) ** I've updated the search for recent changes, added a new one for patches, and added one patch as a trial.
 * 32) *** Thanks. Subject to resolving these "access" and "visibility" issues then, I assume Patches will be on the way out. We'll need a new link to a page that just has the "Audacity Patches" product. Assuming it's what we want, I added the following to the description of "Audacity itself" in "Audacity Patches" - ""Bug Wranglers" who have a patch for a pre-existing issue in the "Audacity" Product should attach it directly to that issue." The rationale is that we wouldn't want an experienced developer thinking they have to add a patch for a P1 to the Patches product, only for it to be missed or have to be moved into the actual bug.
 * 33) *** Good. Makes sense.
 * 34) *** Gale: Where should we archive issues that were raised in the Audacity Patches product? I archived one of those by treating it as I would a bug (moved into the Audacity product with a rating and OS definition); I see you resolved one while keeping it in Audacity Patches. Since the Patches product isn't actually a measure of how many issues were raised outside of a pre-existing numbered bug, my current feeling is that archiving them in Patches may not be as useful as in Audacity. Is there a good reason to archive in Patches?
 * 35) *** JC: I am easy with either policy. You choose.  I think it will be disconcerting for someone with their first patch to go to a patch tracker and find no patches at all (because they have all been moved to the Audacity product).  That's an argument for keeping patches in the patches product and not moving them.  However, to try to get the best of both worlds the Patches search now finds patches on both Audacity and Audacity Patches, so it's less of an issue if we do move them.
 * 36) *** Gale: I guess I was really figuring to replicate the Patches page - so if someone went there and found no patches because they had all been applied or rejected, that would be correct. If we reject a patch ("resolved -won't fix" meaning "won't apply"), then in some ways it would be better kept in the Patches product; but if we like a patch and attach it to a bug in the Audacity product, we must I assume move it into that product or make some flag that it's been moved. Really, this is a mess where the Wiki was crystal clear and I think we shouldn't move more patches into Bugzilla at the moment until we've thought more. I think a database for patches is better, but in practice I'm not sure Bugzilla is a suitable database when you look at it in detail.
 * 37) *** JC: I think your thinking is right (if we have a patches product). Move good patches and don't move ones we decide not to apply.  At least (now) the patches search will find patches whichever product they are in.  What is nice about the wiki is that we can give some instructions there too and update them easily.  The volume of patches is sufficiently low that it is still all very manageable on wiki.  People like Ed and Benjamin who offer many patches could have bug-wrangler rights and add patches directly into Audacity product - usually as new issues.  People who only ever offer one patch could be 'sandboxed' to the wiki.  This would certainly keep the wiki table small.  We'd remove the patches product as there'd be no call for it.  Thoughts?
 * 38) *** Gale: Yes that might work, but I still don't especially like creating "Audacity" issues for "Patches to be triaged", just on grounds of *quantity* of patches offered. My preference would be "all Bugzilla" or "all Wiki", and live with table formatting and commenting issues if we choose Wiki - Wiki is not hard to keep clean, unlike Bugzilla. I think what "might" work on Bugzilla is to not use its own "move between product" feature at all. Keep everything in Patches. If a Patch is "resolved-fixed", that means we applied the patch. If prior to that, it was attached to a pre-existing Bugzilla bug (or exceptionally, an Admin deemed a new issue should be created for it), it remains in the Patches product as "Open", but we duplicate the patch into the bug in the Audacity product, so replicating what we would do if we had Patches. We just have to remember to close the bug in "Patches" when we close it in "Audacity". You can attach a Bugzilla URL to a bug that will download the attachment. You can get the URL by right-click > Copy link address. If a URL is attached, you just have to click "Details" in the attachment to get the actual file you want to download. Or, just use one of the URL fields to give the download link, so making it clear this comes from the Patches product.  Note: attaching to Bugzilla is still not IMO as nice as Wiki, where you can view the file in the browser. Can we remove the Bugzilla "security restrictions" that prevent viewing text files? And I'm still unclear about the difference between "URL" and "bug URL" in the bug editing screen. Unless we use "bug URL" for attaching a link, I'd get rid of that.  Anyway, I think we have two choices - "all Wiki" or "all Bugzilla". Either would work for me. I think I marginally prefer Bugzilla, but would like your view.
 * 39) *** JC: as a general rule having replicated info is a bad move. I'd be inclined to try to avoid duplicating issues.  I don't think it's going to matter hugely if we do though, in this case.  You marginally prefer bugzilla, so lets use it for patches, and am happy to follow whatever protocol around copying you're happy with.
 * 40) *** GA: I said I preferred Bugzilla before, but pulled back when I looked at the complications. I think there are too many problems with creating issues in Audacity product for every patch, and too many problems moving them between products (e.g. confirmation screens,patch submitter gets confused as to where his patch has gone, submitters who aren't wranglers can't comment after the issue is moved to the Audacity product). So I think if we consider the Patches product as just a patches-for-triage list, and not a bug list at all, it might work. I think we should try with Bugzilla with the next few patches, but on an experimental basis. Like you say, we probably we want to nurture/educate submitters so that we can move them to wranglers sooner rather than later (in which case there isn't "duplication"), but try to avoid unnecessary creation of issues. For example, I would expect wranglers to use initiative to still post patches to "Patches" (or -quality) if they had a patch for a small issue which may be better treated as part of a proposal; to write to feedback@ if they found a typo; to post on Wording if they had an idea for a self-contained wording change. These are just examples of "better" approaches for those cases.
 * 41) ** It is easy to move bugs between products. Change the product id in the edit screen.
 * 42) Documentation Checklist "for documentation changes needed for Manual and any developer recruitment drive" - I think the unofficial [Audacity-Manual] list and/or Manual Talk pages are OK for the former, and the latter needs its own page(s) if/when it happens. So this could go.
 * 43) Wording, where we track suggested changes to the exact words that appear on a dialogue - Barely used now.  It was mainly used to track typos/minor rephrasing of text for existing code, while waiting for someone with write access to do the necessary. There seems little appetite to discuss these, and GA now just makes obvious (or "reasonable") string changes. Possibly worth retaining for any one-off changes that may need discussion, but are too small to be a proposal. Clearly -devel should not be used. -quality is an alternative, but harder to track once the thread dies?
 * 44) Pending website changes where we post suggested wording or layout changes to our audacity.sourceforge.net and audacityteam.org sites (except the Wiki). Barely used. GA currently does necessary web changes under do-ocracy, except for changes sufficiently sensitive that they need Team discussion. Useful comments on the web site are received from the public, but via feedback@. I think this page can disappear.
 * 45) I have started to draft (offline) a page to store Proposed logos, icons and skins. Users often send us "improved" or "more modern" logos and icons, or point to (often temporary) pages that have them. There is a strong case for being consistent in graphic design, so that software and web sites make matching simultaneous changes, but we should have a page to store what we get sent.
 * 46) * JC: +1


 * JC: I do think the Bugzilla message of the day should get a link to a wiki page which gives a lot more information. I would like to see the explanation of P1-P5, how to get rights to edit, something about what summary and review issues are and so on.  There is no hurry to get that material, but we should have it.
 * Gale: I assume that could all work on Bug Lists. It already has the rating definition, but most wranglers should post as Px anyway. We still haven't decided *where* someone writes to so as to send their first patch and get access, or to ask for bug wrangler rights. I think it needs to be a non-subscription address, so -quality is out. I'm prepared to be the recipient, or you can be, but let's decide. Also I think "Summary" and "Review" are hazy as already discussed. Either they should be "flags" or we need more classifications like "Repeatable - Review". "Review" bugs will likely not be dealt with, as you suggested elsewhere.


 * JC: I'm also aware of conflicting pulls with regard 'rules and regulations'. It is essential that working on Audacity be fun for people, so we somehow need to get across that these are things that save everyone time rather than red tape that costs everyone time. We have to be quality conscious and at the same time sufficiently relaxed about it so that no one feels that it's just hard work - even though hard work is involved.  A tough call.
 * Gale: Indeed, I've already seriously proposed a couple of times that we could just go back to the old, much looser bug tracking. There may be another conflict between "quality" and releasing a major version at all. If I decided not to bother with bug tracking any more, we may well become less rigorous anyway, but at least we've got a better infrastructure in place now where others could take over. Is the problem just cosmetic - we need to dress the rules up in more relaxed clothing?

Old Guidelines for use of Checklist

 * Follow the existing layout/style:
 * No colours (some people objected to red for P1's which we did before)
 * Start with a one sentence summary of the issue in bold.
 * Add further succinct information if required. Place "steps to reproduce" in the summary sentence if possible, and link to further information rather than obscure the list with it.
 * P1 Release Blockers and P2/P3 release-noted issues: Decide if the issue is a P1 Release Blocker, as described below: this is an empirical rather than a hard-and-fast process. If the item is not deemed a release blocker, then decide if the issue is important enough to release-note, should it remain unfixed at release time. If yes, these items are automatically P2 or P3. If not, they are P4 or P5.
 * How to decide if an item is P1:
 * A crash is often P1, but regressions are usually P1 (or P2) even if they are not crashes, unless of minor usability impact. A regression gives a user reason not to upgrade.
 * Not all crashes are release blockers. For example, a crash bug due to a known driver problem with a particular device would tend to be P2/P3 while being assessed. If the assessment found no code changes that would help, the issue would be moved from the Checklist to Known Issues pending release-noting.
 * Non-crashing, non-regression bugs can be deemed as P1. This depends not only on the impact on usability and the number of users affected, but also on the ease and benefits of fixing compared to leaving "as-is".
 * Bugs initially allocated to lower priorities may be promoted to P1 if a subsequent assessment finds them easy to fix (or demoted from P1 to a release note if they can't be fixed without disproportionate effort).


 * The current de facto standard is that Gale decides on the P numbers for a bug, subject to lobbying from developers.
 * We expect to be rewording the bug title as it becomes clearer what the real problem is.
 * We expect to sometimes solve part of a bug and hence need rewording and/or rating changes.
 * Bug ratings can go up or down.
 * Deciding between a P2 and P3 rating can be difficult when it is a moonphase bug that is suspected as a symptom of some deeper malaise and/or the standards wanted for a non-Beta release may themselves be subject to interpretation.

Solved P2: Ctrl+Mouse Wheel causes Crash (hang)
Moved to Ctrl_Wheel_Crash.

Solved P2 Custom FFmpeg Export Options window oversized
LRN found no apparent way to control widget size with ShuttleGui. He shortened descriptions and textboxes, and MJS committed further changes, but dialogue still too wide on Ubuntu (and even on Windows if a non-default font face with larger characters was used).

LRN proposed:
 * 1) Intensive ShuttleGui hacking
 * 2) Restrict window size to available desktop area, add scrollbars to scroll around
 * 3) Scrollable dropdown lists
 * 4) Build the dialog around a few hideable groups of widgets (3DS Max comes in mind). ShuttleGui doesn't support this out-of-the box
 * 5) Single big (scrollable) treeview widget with all the options, turn codec and format lists into drop-down lists (probably residing in the same treeview).

JC came up with changes that solved the problem.