Talk:New Builds Team

From Audacity Wiki
Revision as of 15:59, 19 February 2017 by Galeandrews (talk | contribs) (Reply to James)
Jump to: navigation, search
  • Use blogs to coordinate this, e.g. the Calls and the tracking of user response?
  • Want high payoff for testers - they get early access to features that interest them?
  • What level of participation by testers makes what level of effort by us in setting this up worthwhile?

Lots to work out...

Gale 06Jul16: I would not like to track and co-ordinate in the WordPress interface we have now. And if we want any tester input there, it's ruled out because we can't give access. So really the blogs are only for calls and announcements, I guess. And I still think we should use the announcements e-mail system as well. Who can find the "blogs" on our current WordPress site?

James 06July16: Tracking is more easily doable than you think and similar in ease to tables in Wiki. We could even give a few alpha testers non publishing rights on Wordpress, I believe, but let's not plan to. Blogs can't currently be found because we don't have any written, other than announcements. Blogs can appear in sidebar and be very findable.

  • Gale 16Feb17: Sidebar news/blog entries are currently hidden below the scroll (or at the bottom of the page if zoomed in or using mobile).

Different levels of engagement

Gale 16Feb17: Users interested in trying out new features in an experimental release probably have a little less engagement than those willing to test more broadly in a milestone alpha (a.k.a. KTT alpha). Both will have much less engagement than anyone willing to test nightly builds on a regular basis from the start of the development cycle, but we should try to encourage a few users into that too.

Nomenclature Alpha/Beta

Peter 16Feb17: James wrote on the page: "This is for selected alphas, ones that could be suitable for production use."

Tradtionally, and customarily, such alphas that are considered "suitable for production use" are labelled as Betas.

Our Alpha nightlies are usually bleeding-edge works-in progress.

At risk of opening up an old Audacity discussion about "no Betas" I really think we should be considering forming not an "Alpha Test Team" but rather a "Beta Test Team" - it would appear more honest and more clear.

IIRC the reason we adopted the "no Betas" was because of the long run of the 1.3.x "Beta" series without getting anywhere near a formal release. This should not be the case if we adopt what I am suggesting here.

See this Wikipedia page on Software release life cycle

  • Gale 16Feb17: I suppose it is possible that we could call an alpha at semi-freeze (which typically has no open P1's) a "Beta" however there is as you say the historical baggage of our Betas and the perceived instability that comes with them. I notice that many users call our alpha nightly builds "Betas" and clearly don't understand the difference between the two stages.
  • Gale 17Feb17: My vote is for alpha-milestone-<number> for milestones and for topical alphas, <alpha-<feature>-<number> and not to split the testers initiative into two parts (at least, I think that right now).
  • James 17Feb17: Fine by me for the file-name of the build on Audacity-devel.html, even if a build does happen to be of very high quality. We may not know for sure it is high quality until later.
  • Gale 17Feb17: Do you or Peter think the semi-freeze build only should be deemed a beta? That seems closest to me to the idea of the version cycle. Thence onwards nightlies might stay as beta according to About Audacity. I'm not sure though given my points about user perception/understanding of "beta".
  • James 17Feb17: Your observation that many users conflate alpha and beta, leads me to recommend we settle on just saying alpha. Perhaps in the context of Audacity-devel.html it is even enough to say 'build'. It is clear they are not final Audacity releases.
  • (Peter 18Feb17: It seems to me that there are three types of testers that we need to recruit and maintain:
    1. A-testers: these are folk who test nightlies and in particular test the (often small) changes that are introduced in each nightly, for example bug-fixes.
    2. B-testers: these are folk who we would want to deploy when big changes are rolled into Audacity. A good upcoming example could be when we fold James' new menu structures into Audacity. A past example could have been scrubbing in its later implementation days. A good cadre of B-testers could prove useful then.
    3. C-testers: these are folk who would test an Audacity build in its entirety, typically using their normal workflows.
In the software industry the A-testers typically worked with alpha releases, the B-testers were typically provided with Betas and the C-testers had Release Candidates or Kick-The-Tyres builds. Whether we have to ability to recruit and maintain such complexity is an issue for discussion I feel.
  • James 18Feb17: I'm with Peter on this. I think our best way to attract testers is through tasty new auteur builds that give previews of possible new features. These attract C-Testers, people using it in their production work (with a little more care around the newest features). Some of those will get interested in a particular feature and be keen to try out iterations (B-Testers) and some will also learn of Audacity test approach from being a C-Tester and decide they want to be more involved by being an A-Testers subscribed to [email protected] C-Testers and B-Testers are in my view the responsibility of the auteur. It's in the auteur's interest to use this feedback and engage with these testers. That's not to say QA can't be involved. I would advocate a single board on the forum for C & B builds. Auteurs will maintain a page on wiki about their builds, and may update with news. Occasional blog posts with screenshots will be made by auteurs and reshared on facebook. To convert more C & B testers to A, auteurs and QA can do the occasional blog post about bugs in HEAD found and fixed via C and B testing.
  • Gale 19Feb17: Some of us are opposed to "auteur" builds from outside the Audacity project. That is what the upcoming vote is about. We can't have an integrated Audacity(R) test program IMO with third-party projects involved - not until Team approves it anyhow.

    IMO we're playing down new features in HEAD too much with the suggested approach - those features likely will be in the next release. New features in "auteur" builds will I assume mostly come on stream in the release after next (if at all). Otherwise, what is the essential difference between "auteur" builds and a well chosen alpha pre-release from HEAD, other than "auteur" branding and distancing from the real Audacity(R) project? Is the difference a very slightly earlier preview of what will be in the Audacity(R) alpha pre-release builds? If yes, we should be careful of providing too many similar builds to test.