ToDo: Pdf of Manual

From Audacity Wiki
Jump to: navigation, search
Automation Progress: Pdf of Manual


  • Done.png Preliminary exploration of what tools are needed/available
    • Done.png Reportlab (open source version) proved to be too limited. Rejected.
      • In particular it will not do the page breaking for you. Instead you create canvases and draw to them, so you have to work out in your own code the size of everything.
      • wxPdfDoc has similar limitations. It is a port of the PHP FPDF library, and low level. You are responsible for item positioning.
    • Done.png QPDF explored. Rejected.
      • QPDF is for low level PDF manipulation.
      • QPDF is good for splitting/joining pdf documents.
      • Because pdf is already a low level format QPDF is not suitable on its own. It would need to be used with something like Reportlab and reportlab won't do the sizing calculations.
    • Done.png LaTeX path explored and settled on.
      • LaTeX gives us a 100% text based file format.
      • LaTeX is a fully professional document typesetting solution.
      • LaTeX has an excellent stack-overflow like Q&A site.
      • LaTeX is also the intermediate file format used by doxygen in going to pdf. This may help if we write a manual for developers at some future date, as class and in-code explanations are already extracted.
      • I'm currently using MiTeX 2.9 for Windows (which includes pdflatex) as the LaTeX implementation package.
      • pdflatex is well maintained and has sufficient coverage of the pdf features we need
      • I'm using TeXMaker 5.0.2 for Windows (it's a LaTeX IDE).
      • TexMaker can easily/quickly preview results which makes this LaTeX set up a good typesetting development environment.
  • Done.png Extraction of raw-but-template-expanded text from our wiki via API.
    • The key step was determination of our endpoint.
  • Done.png Preliminary ordered listing of pages to extract
  • Done.png Preliminary adjustments to alphamanual content to make it suitable for a book. For example:
    • New page for Karaoke rather than buried in a menu page.
    • Adjustment of Your-First-Recording to make it more suitable for an introductory piece of text.
  • Done.png Work getting the manual into a good shape
    • Done.png Upload raw Light theme button images (without the buttons) and make sure manual is using them.
    • Done.png Create a template for toolbars that in particular adds the required grey border so that toolbar boundary does not disappear into the page.
    • Done.png Recreate full and standard shortcut listing pages. Add fullshortcut template to allow one page to fulfill both roles.
    • Done.png 2.2.0 Preferences/Menus images and image-maps and effect dialog images regenerated. Over 480 revised images uploaded.
  • Done.png Scripted production of manual pdf
    • Done.png Got feet wet with pdflatex using Mitex for Windows and TeXMaker
      • Bug: sync files can stay open and may need deleting.
      • Bug: TeXMaker may hang. When it does, it and pdflatex need to be killed by task manager, even though its window is closed. Otherwise TeXMaker won't launch.
    • Done.png Basic experiments with page layout and page breaks. We can set margins, columns, and inter-column spacing.
    • Done.png Writer2Latex plug-in for LibreOffice found and explored.
      • Importing html and exporting LaTeX gives a LaTex intermediate format that is editable and experimentable with to get best results.
      • A number of bugs with the writer investigated and worked around:
        • Bug with parameter count in macro @Hy is caused by URLs with anchors in them. The # in the anchor must be escaped with a \. Writer does not do this.
        • Bug with \newline when line not started can be worked around by putting ~\\ on the line before the \newline.
        • Writer2Latex shows us some of the formatting tricks (LaTeX equivalents) needed to get html into pdf format. However it works from odt, not direct from html, and is not suitable for our eventual automation plan.
    • Done.png Basic experiments with html2tex by Frans Faase.
      • Done.png Have it compiling/running under MSVC.
        • Fixed deprecated warnings and wchar_t issues.
        • Fixed stack overflow bug (STACK_SIZE was not checked)
      • Done.png First runs to create a usable pdf doc from NewFeatures.html.
        • Images and tables were not available
        • Hyperlinks come out as footnotes.
      • Done.png Add in other pages, to show how chapters can be combined.
      • Done.png Second stage to creating a usable pdf doc from NewFeatures.html using further modified html2tex.
        • Images included, and sized correctly.
        • Link information available.
    • Done.png Mangling of html, removal of sidebar and contents div (beautifulsoup).
      • My attempt to do this using html2tex showed up limitations in its design. It's a two-pass 'as you go' converter, with very limited ability to do structural edits.
      • Investigated and created mangling code, in python, using beautifulsoup.
      • Done.png Automatically remove all content we don't want from all files.
      • Done.png Automatically fix up the bad-bracketting (may not be needed).
        • Turns out it was needed, but not the full Rolls Royce solution, just specific work arounds.
      • Done.png Remove need for html2tex, by doing its mangling in Python/beautifulsoup instead.

Stage 1

Stage 2

Stage 3

Stage 4

Stage 1 - No Images. No Links

Stage 2 - With Images.

Stage 3 - With Images and Links.

Stage 4 - With ImageMaps too.

Ragged ends

Even ends

Before - Ragged ends to chapters.

After - Fixed.

  • Done.png Fix up import issues
    • Done.png Work around MediaWiki tag order being broken, so removed ednotes break unordered lists.
    • Done.png Fix warning symbols being treated as inline (character) elements, which look ssilly
    • Done.png Collect several chapters into one - move all chapters down one level.
    • Done.png Fix toolbar images coming out too far across page.
  • ToDo.png Lots of fine-tuning
    • ToDo.png Remove forward and back links e.g. in FAQ, which look silly
    • ToDo.png Generate document references index using latex indexing.
    • Done.png Abolish the 'alpahabetic' chapters by moving their topics into better places.
    • Done.png Fix titles with images in them look fine in the document but funny in the toc.
    • ToDo.png Fix Glossary entries don't link to wikipedia and icon too close to the text.
    • ToDo.png Nice boxes for intros and warning.
    • Done.png Special casing for a very few larger images (e.g. Toolbars overview).
    • Done.png Special custom front page, based on current image.
    • ToDo.png Wiki-Template to exclude certain content from the pdf.
  • ToDo.pngIndexing and Toc
    • Index for all menu items
    • Index for all glossary terms
    • Table of contents we get 'for free', because we are using LaTeX.
    • Ctrl+F Searching we get 'for free', because we are using text rather than images.

Automatic Table of Contents

Table of Contents as seen in pdf reader


Searching for 'Selection' in the pdf manual

  • Done.png Complete LaTeX error-free import of all pages, bar anomalies
    • Done.png Tour Guide
    • Done.png New Features and FAQ (top level)
    • Done.png Entire FAQ
    • Done.png All E's (Effects, Edit and Ext*) excluding two anomalies:
    • Done.png anomalies Edit_Toolbar and Ext_Bar_Menu_Tools
    • Done.png All F's
    • Done.png All Tutorials
    • Done.png All T's
    • Done.png Glossary
    • Done.png anomalies recording_length and track_control_panel_and_vertical_scale
    • Done.png index.html page. This needs special customised version.
    • ToDo.png quickstart page. This needs special customised version.
    • Done.png Z, O and G
    • Done.png All remaining letters.
  • Done.png Image conversion
    • Done.png Automatically include images (more tag handlers needed).
      • Done.png Layout tweaks for images.
    • Done.png Got feet wet with 'PIL' the python image manipulation package. (Pillow distribution).
      • Done.png Size tweaks for images.

Example: Bad tooltips

Tooltips aalways open

Bad Tooltips - All tooltips open

  • Done.png ImageMap conversion
    • Done.png Proof of concept using {stackengine} and {hyperref}. We can now hyper-link from rectangular areas in an image using a script to specify the regions.
    • Done.png Convert image-map links from proof-of-concept to actual doc.
      • Done.png href for single area.
      • Done.png multiple hrefs.
      • Done.png hrefs actually linking.
  • Note that tooltips are a separate feature
  • PDF readers do not consistently support tooltips.
  • I can make tooltips that work in Adobe Reader, but in Chrome they always appear (and look terrible) and in FoxIt they only appear when you click.
  • Conclusion: It is better not to do them. If there was a lot of demand we could do a with-and-without version.

Bad image sizing


Before - Image size and position wrong

After - Fixed


  • Nov 16 2017 - DraftManual2.2.0_v02.pdf available. This is the whole manual, except 4 pages. It looks basically good, though there are minor formatting issues. The live table of contents is a win.
  • Nov 17 2017 - DraftManual2.2.0_v03.pdf available. Now formatted using Helvitica (sans) font rather than Times New Roman. Some unicode issues addressed.
  • Nov 19 2017 - DraftManual2.2.0_v04.pdf available. Custom front page. Imagemaps working.