Automation Project Progress

From Audacity Wiki
Jump to: navigation, search
This provides a checklist of progress on the automation project


Driving Audacity by an external script:

No progress on improving this yet.

  • ToDo.png Get working again and in Python (earlier demos were in Perl mostly)
  • ToDo.png Error handling for new commands
    • ToDo.png Design the strategy
    • ToDo.png Write
    • ToDo.png Document
  • ToDo.png New Audacity commands (for test scripts)
    • ToDo.png Design/refine commands and parameters
    • ToDo.png Write
    • ToDo.png Document

Image Script

Collecting / Modifying Images for the Manual:

This was harder than it looked!

  • Scripting the imagemaps gives much better hit-boxes (consistency) than 'doing it by hand'.
  • wxWidgets menus cannot be auto-captured with normal tools. There is no way to make them drop down programmatically. Instead the menus were completely redrawn using code.
  • ImageMagick is perfectly OK for scripted cropping and image manipulation.
    • Since using Imagemagick I have found PIL(llow) for python which IS more flexible image manipulation and better syntax.
  • Done.png Very basics of capturing effects dialogs automatically.
  • Done.png Menu Image Maps. Generation of the images, the image maps, tooltips and links.
    • Now includes the top bar.
    • Now hides the ext menus, unless we are showing Ext-Bar or Ext-Command.
    • Optionally generates template of menu description pages.
    • Added customisable Redirects so we can go to arbitrary pages.
    • Added support for creation of command list
    • Added support for standard and full key-binding information.
    • Added support for extra annotations.
  • Done.png Capturing preferences panels automatically.
  • Done.png Script to crop and upload 190 images. Boy that would have been tedious by hand!
    • Using image Magick, magick mogrify -crop 1000x1000+101+0 -path ../Output *.png
    • Then use python pwb upload2 C:/dir/Output "Cropped Track Panels"
    • The 'easy' way to get the images was to create three wiki pages for the different croppings, and then from a browser save-web-complete
  • ToDo.png Annotation script
    • Done.png Reasonably fast by-hand Inkscape image-layers approach found and used to date.
  • ToDo.png Python (or other) test script to generate all manual images
    • Done.png Menu category (external program)
    • Done.png Button category (using theme file-export)
      • ToDo.png Button mogrification (e.g. showing different button states)
    • Done.png Toolbar category (using screenshot-tools)
    • ToDo.png Dialog Category
      • Done.png Effects Dialogs (using screenshot-tools)
      • Done.png Preferences Dialogs (using screenshot-tools)
      • ToDo.png Other Dialogs
    • ToDo.png Waveform Category
    • ToDo.png Add timing info to output


Downloading / Modifying / Uploading from Manual by Script:

This works

  • The main tool, PyWikibot works well enough.
    • It is a bit slow, probably due to server speed and throttling.
    • We can upload about four pages a minute using it, but that is fine as it can just be left running until done.
  • Use in a simple way for bulk uploads.
  • Use custom web pages and get-web-page-complete from within browser to download lots of images.
  • Done.png Got feet wet with PyWikibot.
    • Use Python 3.6, as 2.7.x has unicode issues with some scripts. On wiki use special pages to set up a bot password. Authentication set up and working.
  • Done.png PyWikibot upload2:
    • Now can upload multiple image files 'whilst having a coffee'. Over 100 image files uploaded. No longer necessary to baby sit.
  • Done.png Pywikibot replace:. 'fixes' are their term for patterns used to make changes.
    • Done.png Custom fixes added for replacing spans and lists with our equivalent templates.
    • Done.png Custom fix done to introduce i18n template in place of the flags. Script ran through 1068 pages, once to replace and once to consolidate running for 8 hours in total.
    • Examples and comments in PyWikibot replace are in Hungarian.
    • Getting the fixes working right is mostly about correct escaping. You must use r strings, and it is vital to escape [ ] | ( ) characters.
    • For our use we typically need the (.*?) weak repetition in wildcards rather than strong.


New Commands Available to Nyquist Too:

Reading online

  • SWIG is well maintained and currently looks suitable.
  • Am following SWIG commits on GitHub
  • ToDo.png Transfer new commands to Nyquist as 'foreign functions'.
    • ToDo.png Make swig header files for the new commands.
    • ToDo.png Get swig working properly with embedded nyquist.
    • ToDo.png Develop and test.
    • ToDo.png Document.

Whats is That? Page

Dynamic Javascript 'Image Map' for Audacity:

One of the early take-homes from experiments is to do relatively little image updating with hover movements. This is to avoid distracting the viewer with things happening in their peripheral vision.

  • Do most substantial updates on a click.
  • A key difference from the wiki image map is that this map has multiple levels.
    • That in turn means a lot more content is connected, and also a lot more hit-targets to define.
  • Done.png Initial experiments with javascript tools.
    • Likely to use canvas.
    • Repainting canvas completely looks to be fast enough that we don't need to be smarter.
  • Done.png Proposal page set up with some mock ups.
    • Done.png Design concept for annotation interaction properly worked out.
  • ToDo.png Coding
    • Done.png Png images captured for the very very basics.
    • Done.png Image and manual-contents iframe positioned and sized.
    • ToDo.png Reset button.
    • ToDo.png Capture local copies of wiki/manual html for right hand side-strip.
      • Done.png Construct new top-level html side-strip content.
      • Done.png Convert 20 Sample html pages to format for side-strip
      • ToDo.png Convert remaining html pages to format for side-strip
      • Done.png Convert 303.css to narrow format
    • ToDo.png Spec files.
      • Done.png Spec file containing the Audacity menu information.
      • ToDo.png Spec files for annotation locations.
      • Done.png Spec file for main toolbar areas.
      • ToDo.png Spec file for sub areas.
      • Done.png Spec file for image cache.
      • Done.png Auto generate caption list from spec file.
      • ToDo.png Compute slide location from area location.
    • ToDo.png Implement effects.
      • ToDo.png Implement smooth scroll-to behaviour.
      • ToDo.png Implement annotation fades.
      • ToDo.png Implement annotation arrows
      • ToDo.png Implement highlight boxes.
      • ToDo.png Implement transition effect for full-page help.
  • ToDo.png Automation
    • ToDo.png Python script for html file conversion for side-strip (using beautiful soup approach)
    • ToDo.png Python script for spec file creation (initial work is semi automated, semi by hand)
  • ToDo.png Hosting
    • Done.png Name of subdomain discussed and agreed.
    • ToDo.png Agree temporary place to showcase.

Pdf of Manual

Wiki Manual as a PDF:

Snag list for this project.

This has involved work with a large number of tools. Key take aways:

  • Many of the pdf tools are low level and poorly maintained. I ended up rejecting / abandoning quite a few that seemed to be more work than they should have been.
  • Use LaTeX as an intermediary.
  • Do document restructuring using HTML tools. Beautifulsoup is mature and well designed.
  • Do most of the heavy lifting (page layout, cross referencing and detailed format tweaking) in LaTeX. LaTeX is designed for producing documents.
  • Generate and output both .html and .tex files as we process the source html into latex.
    • The .html isn't used in generating the pdf, but it makes debugging much easier.
  • Done.png Preliminary exploration of what tools are needed/available
    • Done.png Reportlab (open source version) proved to be too limited. Rejected.
      • In particular it will not do the page breaking for you. Instead you create canvases and draw to them, so you have to work out in your own code the size of everything.
    • Done.png QPDF explored. Rejected.
      • QPDF is for low level PDF manipulation.
      • QPDF is good for splitting/joining pdf documents.
      • Because pdf is already a low level format QPDF is not suitable on its own. It would need to be used with something like Reportlab and reportlab won't do the sizing calculations.
    • Done.png LaTeX path explored and settled on.
      • LaTeX gives us a 100% text based file format.
      • LaTeX is a fully professional document typesetting solution.
      • LaTeX has an excellent stack-overflow like Q&A site.
      • LaTeX is also the intermediate file format used by doxygen in going to pdf. This may help if we write a manual for developers at some future date, as class and in-code explanations are already extracted.
      • I'm currently using MiTeX 2.9 for Windows (which includes pdflatex) as the LaTeX implementation package.
      • pdflatex is well maintained and has sufficient coverage of the pdf features we need
      • I'm using TeXMaker 5.0.2 for Windows (it's a LaTeX IDE).
      • TexMaker can easily/quickly preview results which makes this LaTeX set up a good typesetting development environment.
  • Done.png Extraction of raw-but-template-expanded text from our wiki via API.
    • The key step was determination of our endpoint.
  • Done.png Preliminary ordered listing of pages to extract
  • Done.png Preliminary adjustments to alphamanual content to make it suitable for a book. For example:
    • New page for Karaoke rather than buried in a menu page.
    • Adjustment of Your-First-Recording to make it more suitable for an introductory piece of text.
  • Done.png Work getting the manual into a good shape
    • Done.png Upload raw Light theme button images (without the buttons) and make sure manual is using them.
    • Done.png Create a template for toolbars that in particular adds the required grey border so that toolbar boundary does not disappear into the page.
    • Done.png Recreate full and standard shortcut listing pages. Add fullshortcut template to allow one page to fulfill both roles.
    • Done.png 2.2.0 Preferences/Menus images and image-maps and effect dialog images regenerated. Over 480 revised images uploaded.
  • Done.png Scripted production of manual pdf
    • Done.png Got feet wet with pdflatex using Mitex for Windows and TeXMaker
      • Bug: sync files can stay open and may need deleting.
      • Bug: TeXMaker may hang. When it does, it and pdflatex need to be killed by task manager, even though its window is closed. Otherwise TeXMaker won't launch.
    • Done.png Basic experiments with page layout and page breaks. We can set margins, columns, and inter-column spacing.
    • Done.png Writer2Latex plug-in for LibreOffice found and explored.
      • Importing html and exporting LaTeX gives a LaTex intermediate format that is editable and experimentable with to get best results.
      • A number of bugs with the writer investigated and worked around:
        • Bug with parameter count in macro @Hy is caused by URLs with anchors in them. The # in the anchor must be escaped with a \. Writer does not do this.
        • Bug with \newline when line not started can be worked around by putting ~\\ on the line before the \newline.
        • Writer2Latex shows us some of the formatting tricks (LaTeX equivalents) needed to get html into pdf format. However it works from odt, not direct from html, and is not suitable for our eventual automation plan.
    • Done.png Basic experiments with html2tex by Frans Faase.
      • Done.png Have it compiling/running under MSVC.
        • Fixed deprecated warnings and wchar_t issues.
        • Fixed stack overflow bug (STACK_SIZE was not checked)
      • Done.png First runs to create a usable pdf doc from NewFeatures.html.
        • Images and tables were not available
        • Hyperlinks come out as footnotes.
      • Done.png Add in other pages, to show how chapters can be combined.
      • Done.png Second stage to creating a usable pdf doc from NewFeatures.html using further modified html2tex.
        • Images included, and sized correctly.
        • Link information available.
    • Done.png Mangling of html, removal of sidebar and contents div (beautifulsoup).
      • My attempt to do this using html2tex showed up limitations in its design. It's a two-pass 'as you go' converter, with very limited ability to do structural edits.
      • Investigated and created mangling code, in python, using beautifulsoup.
      • Done.png Automatically remove all content we don't want from all files.
      • Done.png Automatically fix up the bad-bracketting (may not be needed).
        • Turns out it was needed, but not the full Rolls Royce solution, just specific work arounds.
      • Done.png Remove need for html2tex, by doing its mangling in Python/beautifulsoup instead.

Stage 1

Stage 2

Stage 3

Stage 4

Stage 1 - No Images. No Links

Stage 2 - With Images.

Stage 3 - With Images and Links.

Stage 4 - With ImageMaps too.

Ragged ends

Even ends

Before - Ragged ends to chapters.

After - Fixed.

  • Done.png Fix up import issues
    • Done.png Work around MediaWiki tag order being broken, so removed ednotes break unordered lists.
    • Done.png Fix warning symbols being treated as inline (character) elements, which look ssilly
    • Done.png Collect several chapters into one - move all chapters down one level.
    • Done.png Fix toolbar images coming out too far across page.
  • ToDo.png Lots of fine-tuning
    • ToDo.png Remove forward and back links e.g. in FAQ, which look silly
    • ToDo.png Generate document references index using latex indexing.
    • Done.png Abolish the 'alpahabetic' chapters by moving their topics into better places.
    • Done.png Fix titles with images in them look fine in the document but funny in the toc.
    • ToDo.png Fix Glossary entries don't link to wikipedia and icon too close to the text.
    • ToDo.png Nice boxes for intros and warning.
    • Done.png Special casing for a very few larger images (e.g. Toolbars overview).
    • Done.png Special custom front page, based on current image.
    • ToDo.png Wiki-Template to exclude certain content from the pdf.
  • ToDo.pngIndexing and Toc
    • Index for all menu items
    • Index for all glossary terms
    • Table of contents we get 'for free', because we are using LaTeX.
    • Ctrl+F Searching we get 'for free', because we are using text rather than images.

Automatic Table of Contents

Table of Contents as seen in pdf reader


Searching for 'Selection' in the pdf manual

  • Done.png Complete LaTeX error-free import of all pages, bar anomalies
    • Done.png Tour Guide
    • Done.png New Features and FAQ (top level)
    • Done.png Entire FAQ
    • Done.png All E's (Effects, Edit and Ext*) excluding two anomalies:
    • Done.png anomalies Edit_Toolbar and Ext_Bar_Menu_Tools
    • Done.png All F's
    • Done.png All Tutorials
    • Done.png All T's
    • Done.png Glossary
    • Done.png anomalies recording_length and track_control_panel_and_vertical_scale
    • Done.png index.html page. This needs special customised version.
    • ToDo.png quickstart page. This needs special customised version.
    • Done.png Z, O and G
    • Done.png All remaining letters.
  • Done.png Image conversion
    • Done.png Automatically include images (more tag handlers needed).
      • Done.png Layout tweaks for images.
    • Done.png Got feet wet with 'PIL' the python image manipulation package. (Pillow distribution).
      • Done.png Size tweaks for images.

Example: Bad tooltips

Tooltips aalways open

Bad Tooltips - All tooltips open

  • Done.png ImageMap conversion
    • Done.png Proof of concept using {stackengine} and {hyperref}. We can now hyper-link from rectangular areas in an image using a script to specify the regions.
    • Done.png Convert image-map links from proof-of-concept to actual doc.
      • Done.png href for single area.
      • Done.png multiple hrefs.
      • Done.png hrefs actually linking.
  • Note that tooltips are a separate feature
  • PDF readers do not consistently support tooltips.
  • I can make tooltips that work in Adobe Reader, but in Chrome they always appear (and look terrible) and in FoxIt they only appear when you click.
  • Conclusion: It is better not to do them. If there was a lot of demand we could do a with-and-without version.

Bad image sizing


Before - Image size and position wrong

After - Fixed


  • Nov 16 2017 - DraftManual2.2.0_v02.pdf available. This is the whole manual, except 4 pages. It looks basically good, though there are minor formatting issues. The live table of contents is a win.
  • Nov 17 2017 - DraftManual2.2.0_v03.pdf available. Now formatted using Helvitica (sans) font rather than Times New Roman. Some unicode issues addressed.
  • Nov 19 2017 - DraftManual2.2.0_v04.pdf available. Custom front page. Imagemaps working.
Personal tools

Donate securely by PayPal, using your credit card or PayPal account!