Rake Tasks for Dia to SVG?

[tajmone]

I'm still undecided whether we should handle conversion of the Dia source projects to SVG via Rake or not.

Ideally, Rake could handle:

1. A dia task to build all the SVG images (optimized via SVGO), thus replacing the current conversion script.
2. Thanks to autogenerated file tasks, it would always rebuild SVG images when a their source Dia project has changed.
3. Downloading the correct Dia version and unpacking it somewhere, so it can be used for the task (without interfering with another Dia version present on the end user machine). I.e. basically replacing the current installer script (which is Win only).

The problem here is the strict requirement for a specific Dia version (0.97) which apparently is only available for Windows (see alan-if/alan-docs/wiki/Dia-Diagrams and #12), which would then make the Rakefile unusable on other OSs (including Travis CI), especially if we enforce (2), which would automatically try to rebuild any SVG when its Dia source has changed, or if Rake is invoked via -B, or after clobbering.

Definitely, the Rake tasks will have to check the Dia version before attempting conversion, and skip the operation if the version is not the exact one required.

Before deciding on this, we need to find out if there's a Dia version for Linux and macOS that produces identical SVGs files from the same source projects — i.e. so that no changes are detected by Git after rebuilding an SVG from an unchanged project. If this was the case, we could then perform different version checks based on the OS (we could settle with covering Linux only right now, until we find a macOS user who can carry out the tests for us).

Alternatively, we should find a way to disable the Dia related tasks for those OSs for which we don't have a matching Dia version — this can be done, programmatically, via Ruby code, but detecting the host OS in Ruby requires some hacks that are not bullet-proof (we need to ensure that any Windows, Linux or Mac operating system is detected correctly).

Either way, we ultimately need to decide how to handle this

[tajmone]

Linux Testing

@SicroAtGit, I would need your help regarding the Linux tests to find a Dia version (and its link) that produces identical SVGs to those created by Dia v0.97 for Windows.

The problem is that testing will have to be done locally, since the build script in the repository also optimizes the SVG images via SVGO, which might introduce differences that compromise the tests (another issue altogether, which needs to be considered separately, after this is solved).

Another problem might the actual fonts used, which might not be identical on all OSs, or produce slightly different results because of differences in Dia internal libraries, etc. (not sure, but worth considering).

Overall, I think that managing to cover this task for all OSs and producing identical optimized SVGs is going to be tricky, considered all of the above, but the general problem remains because if different OSs produce differing SVGs, updating the diagrams is going to be problematic (except if either only one maintainer is in charge of this task, or if all SVGs are updated at once whenever there's a change).

The idea of having Rake detect changes to the Dia source and rebuilding the SVG images that require updating is quite important, but if this results in those images being overwritten endlessly every time a contributor force rebuilds all Rake tasks then we'd end up with spurious commits.

What's your view on this? should we just leave this out of Rake, and use the script whenever there's a need to?

Alternative, we could:

• Make Rake just notify when an SVG image needs updating, by printing a warning but not doing anything.
• Add another Rakefile in the Dia sources folder, so it's independent from the main Rakefile, but can still be invoked in that folder — i.e. dropping the current script in favour of Rake, doing the same things but better. Also, this task could initially be Windows only, and as (if) we find solutions for other OSs we can add them in the future.

Combining the two solutions above would already improve the current state of affairs, at least until we can find a perfect solution (e.g. when finally a new Dia release is out for all OSs which handles CairoSVG correctly).

[SicroAtGit]

Can you please provide me a branch where the SVG files were created using only Dia (without SVGO)?

[tajmone]

Can you please provide me a branch where the SVG files were created using only Dia (without SVGO)?

There isn't one, I've added SVGO almost immediately to the script because the optimizations are considerable.

We'll probably have to either open a test repository for this or see if we can manage it via Gists.

I do have a test repository which we can use freely:

https://github.com/tajmone/testing

I'll start by inviting you as a collaborator. If you feel like copying the script over there (without the SVGO optimization part) and a sample Dia project, we could then check if I get identical results. Else, I can set it up later, when I find some time.

[SicroAtGit]

There isn't one, I've added SVGO almost immediately to the script because the optimizations are considerable.

Yes, that's why the question.

For you, it is quickly done to check out from the master branch to a new only-dia branch, comment out SVGO in build.sh, run the build script, commit the changes, and push the only-dia branch to this remote repository.

I can then check out the only-dia branch locally and run my Dia tests. Once Git reports no change in the working directory, I have found a solution that creates a matching result.

If you feel like copying the script over there (without the SVGO optimization part) and a sample Dia project, we could then check if I get identical results.

If you don't like my approach above, feel free to upload the SVG files to your test repository, provide them in a gist, or email them to me instead.

It would be better if I could check with my Linux locally and alone if my Dia tests are successful or not.

Linux (MacOS) support

As I wrote in the other Issues, older program versions are uncommon on Linux. In the package repository there is only one program version, except for a few exceptions, like python:

• Package python (currently corresponds to major version 3)
• Package python2 (currently corresponds to version 2.7.18-4)

With the Dia version and the SVGO version from the package repository, I don't get identical SVG files that match yours. I have installed the fonts we specified, of course.

I installed the specified Windows version of Dia using WINE, adjusted the build.sh to use the WINE Dia version, and added the specified fonts to the WINE environment, but Git continues to report differences in the working directory when I run the build script. The build script continued to run the Linux version from the SVGO tool. So I think maybe the Linux version of the SVGO tool or the version of the SVGO tool is causing the difference.

To make it easier to find the culprit, I would like to focus on the Dia tool first (without SVGO).

If we find a WINE solution, maybe it will then also work on macOS with WINE.

[tajmone]

If you don't like my approach above, feel free to upload the SVG files to your test repository,

I've opted to create dedicated tests on that repository:

https://github.com/tajmone/testing/tree/master/dia

The build script will include in the generated SVGs info about the OS, Dia and SVGO versions used at each run, so we can store all versions even if SVGO is updated. Also, for each execution it will generate two separate SVGs, the plain SVG generated via Dia and its optimized version via SVGO, separately so we can better track if any differences are introduced by SVGO updates.

With the Dia version and the SVGO version from the package repository, I don't get identical SVG files that match yours. I have installed the fonts we specified, of course.

SVGO was updated since I last used it in the repository, and in the past I noticed that often an updated version will produce different SVGs. So it looks like this might introduce a further complication factor, since end users might have not updated SVGO timely.

A possible solution for Rake would be to split the conversion in two steps: first generate the plain SVG files in the Dia folder, then optimize them into the book images folder. The former could be ignored by Git, to avoid cluttering the repository with duplicate SVGs, but this would mean that Rake would always try to convert Dia source to SVGs at first run (since the files would be missing).

On the other hand, if we track the plain SVGs in the Dia folder, then Rake would only run Dia when a .dia project has changed, and SVGO only when their plain .svg has changed. The only real benefit would be the ability to force re-optimizing the SVGs without having to convert Dia sources (e.g. via rake -B svgo).

It's quite an intricate problem, due to all the variables at play here (fonts must be identical, not all Dia versions being available to all OSs, and SVGO being a tool that gets frequently updated).

I'm not really bothered about SVGO — if an image is optimized using a different SVGO version is not really a big deal, it's still optimized, and worst case scenario is that every time someone runs an optimization we end up having to commit all SVG diagrams again. The bad thing would be if two contributors keep using different SVGO versions, all the time, resulting in a bouncing effect of re-writing the same SVGs over and over again, but this should be mitigated by Rake, since SVGO should only be invoked when a Dia source changed, and only for that specific diagram! (again, another huge advantage of Rake over Bash scripts).