Liblouis table for Dutch

[bertfrees]

• Include some official braille code specification document
• Write tests based on the specification
• Validate the existing braille table for Dutch. There is currently a table for the Netherlands and a table for Belgium which are slightly different (and should ideally be merged, see Common table for Dutch liblouis/liblouis#34)
• Improve the existing table or write a new one from scratch

Related issues:

• Common table for Dutch liblouis/liblouis#34
• Liblouis table for mathematical expressions in Dutch #8
• Liblouis table for Dutch without capsign #9

[bertfrees]

@dkager See branch "dkager_dutch"

[dkager]

Thanks. Should we update nl-BE or do a new set of tests for nl-NL? Would be nice to keep them in sync, or to have just one set of tests because these tables are almost identical.
Also, I think task 1 can be marked as completed: we have the 2005 specification of the Dutch braille code (is it online somewhere?).

[bertfrees]

Try extending nl_BE. If all the tests in there are valid for the Netherlands (Dedicon) rename it to nl. Otherwise make a new test file for nl_NL and we can get them in sync again later.

[dkager]

I've reorganized the tests somewhat and added examples from the official documentation. There still isn't a huge lot of data. I'm not sure what else can be added, these tests cover most of the corner cases.

I still have to actually run these, but one omission I can spot right away is the special braille notation for minutes and seconds, i.e.:
The world record time is 3' 5''.
Here you don't use an apostrophe. Instead you write:
⠼⠉⠈⠔ ⠼⠑⠈⠔⠔
I don't think the nl-BE (or nl-NL) table defines this either. It may not be possible to figure this out from the context, because normal apostrophes are used.

[bertfrees]

These are things that liblouis has difficulty with, because it knows little about the context, except for the actual text you give it (and typeform info). You could make a rule that finds occurrences of the pattern "digit-digit-apos-digit-digit-apos-apos" and translate it in a special way. For such simple heuristics this approach works, but if you need to do advanced text analysis, or if you want to mark up minutes and seconds in XML, you need something besides liblouis.

[dkager]

I don't think we get a lot of books where this shows up, so not very high priority just yet.
One more thing that is missing is tests for emphasized text, etc. Will add that and test the whole thing on Thursday.

[bertfrees]

The new data looks very good. It's not a problem if it's not a lot, if it covers everything it covers everything.

I noticed you deleted some tests. In almost all cases you replaced them with an equivalent test but in one or two cases you didn't (e.g. "SLAAT DE VLAM IN DE PAN..."; I know it's kind of the same as "MEER DAN DRIE WOORDEN..." but not completely because the last uppercase word doesn't end with a period). I want to avoid touching other people's tests as much as possible so if there's no good reason to delete a test I think we should leave it.

I'm gonna add back the ascii-output because it's useful for readability. I can do that with a script.

[dkager]

Can you also re-add the all-capitals sentence?

I'm not a big fan of the ascii-input because it's redundant and to me (as a braille reader) the symbols don't make much sense. Could you wait with the ascii-input until I've finished the test data? It makes editing somewhat easier for me as I don't know the ASCII code liblouis uses. Also, I noticed that other harness tests (fi_harness.txt) don't have ASCII?
The Unicode output specification is proving to be very nice to work with, because I can verify it straight away on my braille display! :)

[bertfrees]

I know, that's why I added the Unicode alongside with the ASCII. The Unicode is used by the test harness, the ASCII is for making it readable for sighted people (documentation purpose). I'll wait until you've finished though, then I only have to run the conversion script once.

[bertfrees]

@dkager In commit 1a92520 I've added xfail flags to all failing tests.

[dkager]

"xfail": "Fails because special handling of more than three capitalized words is not supported by liblouis",

So we need support for this that is similar to how emphasized text is handled?

I'll look at the Greek capital letters. They are used in maths so it is important that they work.