14884. Tom 7 (74.109.249.246) – 08 Jun 2024 11:06:40 [ "April" 2024 ]

I added the Widths table, but it turns out there was another issue (which I could only find by stepping through pdf.js in the debugger!): Firefox really wants the "argument" to /FontName to begin with a slash. I guess that makes sense, but none of the validators I found reported any issue here!

FWIW I think the text rendering looks much better in Chrome, but I am happy to have it working in Firefox. Thank you everyone (especially mixmix) for the help and encouragement.

The PDFs on the site should work now. They still don't open in Illustrator; that would be nice to fix but I don't think it'd be a serious obstacle for the main use of the papers, which is to print them out and shred them.

I'll try to make the source code a bit easier to use when I find some more energy.

14883. Tom 7 (74.109.249.246) – 08 Jun 2024 09:34:09 [ Green scream ]

mixmix: I do have a lot to say on all those things and the fact that you're interested is heartening! I am not going to think about making videos for at least 2 days, however.

14882. mixmix (27.127.18.207) – 07 Jun 2024 21:53:44 [ Green scream ]

I really enjoyed the video, Tom! It's such a pleasure to watch you having fun together with your family.

Now, what does it take to make you produce a supplementary, low production cost video on BoVeX (full language description including its type system, semantics, lowering it down to the EL and the IL, and the register machine with its own primops), the boxes-and-glue algorithm, and the PDF generator?

We want to know everything, from how you compile mutually-recursive polymorphic functions through how you traverse the rephrasing tree (any hope for A*?) to how you cope with the missing kerning pairs. What you've tried, what did work and what didn't, what unexpected problems arose, and what's your secret on maintaining sanity while writing a compiler in C++.

14881. jonas (37.191.60.209) – 07 Jun 2024 06:24:41 [ Green scream ]

The video about this project is now online at https://www.youtube.com/watch?v=Y65FRxE7uMc

14880. Elijah (64.64.116.155) – 05 Jun 2024 12:13:10 [ Green scream ]

Super excited for it! Take your time :)

14879. jonas (88.87.242.184) – 01 Jun 2024 16:18:45 [ Green scream ]

Joe: thank you, I was wondering if it was in that repository somewhere, but I'd still like a release too.

14878. Joe (82.69.58.143) – 01 Jun 2024 16:12:03 [ Green scream ]

Is this what you are looking for, Jonas?

https://sourceforge.net/p/tom7misc/svn/HEAD/tree/trunk/rephrase/

14877. jonas (88.87.242.184) – 01 Jun 2024 10:12:40 [ Green scream ]

I'm less excited about the video, because the articles in the robot dance proceedings appear to describe the project well enough. I'd be more happy if you made a source code release of BoVeX, especially of the parts that do not require an expensive video card with 128 GB of RAM. The "Coming soon" message currently at "http://tom7.org/bovex/" is teasing me a bit too much.

14876. TubbDoose (68.149.144.137) – 31 May 2024 23:40:55 [ Green scream ]

Excited for the video!

14875. Anonymous (75.57.29.28) – 31 May 2024 21:55:07 [ ARST ARSW ]

Goodbye Conan link

14874. Tom 7 (104.234.212.16) – 19 May 2024 11:10:43 [ "April" 2024 ]

Awesome, thank you!
I think the Widths table is a likely explanation. That would be redundant with the information already in the embedded font, so it would make sense that some readers would care and others would just use the font metrics. I have the widths from the TTF so it should be pretty easy to add that.

I am working on the video this weekend, so I will try not to let this distract me :)

14873. mixmix (119.244.27.244) – 14 May 2024 23:48:01 [ "April" 2024 ]

OK, so I've spent a bit more time investigating the rendering issues some of the folks are having.

Turns out, there's actually plenty of PDF validators out there. I used one from pdf-online.com whose output looked reasonably constructive. Basically, I was immediately yelled at for a bunch of missing keys, one of which was Widths.

I checked https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/PDF32000_2008.pdf (page 255), and apparently it is really required. I couldn't find what a reader is supposed to do in case of its absence, so I guess it's implementation specific.

Then I went to see what pdf.js (which is what Firefox uses under the hood) does, and to my surprise instead of reading the actual glyph widths from the font data, it relies on BaseFont metrics: https://github.com/mozilla/pdf.js/blob/master/src/core/evaluator.js#L3986

If the referenced BaseFont is not a real thing, it falls back to Times Roman for serif typefaces and Helvetica sanswise: https://github.com/mozilla/pdf.js/blob/master/src/core/evaluator.js#L4046

A fun side note: because Font18 and its brothers (which is what Palatino goes by after https://sourceforge.net/p/tom7misc/svn/HEAD/tree/trunk/cc-lib/pdf.cc#l3808) are not known serif typefaces, nor do they contain the word "serif" in their name, pdf.js treats them on par with Helvetica.

I also was curious why reprocessing your paper seemingly fixes all issues so I compared the original with what macOS Preview exports it as, and it seems like it completely revamps the whole thing and even embeds the kerning values.

Speaking of kerning, it's apparently not very portable to rely on a PDF reader in hope it'd apply it. TIL: it may not and in some cases it absolutely doesn't. I didn't read too much of your code (trying not to spoil the fun), but if you in any way rely on kerning to position the word pieces, I suggest you use the TJ operator instead of Tj.

I hope all these discoveries don't distract you from making the video to which I'm really looking forward!

14872. Anonymous (37.17.237.199) – 14 May 2024 07:55:05 [ "April" 2024 ]

I would pay for an online version of the monofixed version of the algorithm!

14871. Tom 7 (74.109.249.246) – 11 May 2024 11:08:14 [ "April" 2024 ]

Right? I don't get it! I wish there was at least some kind of validator that would tell me "this is broken" instead of just looking wrong on some platforms. :(

14870. Anonymous (37.214.70.215) – 11 May 2024 04:33:09 [ "April" 2024 ]

I can confirm that the version of the paper(s) attached to this post have kerning problems on both android and desktop firefox. Okular shows problems with kerning as well, however the wrong kerning looks different from firefox. Librera FD on android renders correctly. The version in SIGBOVIK proceedings looks fine in every program though. Shurely the "Portable Document Format" can't experience non-portable behavior, right? Right?

14869. Anonymous (91.160.14.162) – 10 May 2024 01:26:53 [ "April" 2024 ]

Great paper(s) as always! No apparent kerning issue on chrome or android.

14868. Anonymous (185.226.120.84) – 06 May 2024 07:31:05 [ "April" 2024 ]

So you are telling me I read through that whole paper with kerning from hell and it was just a display bug???

14867. Anonymous (119.244.27.244) – 04 May 2024 09:58:39 [ "April" 2024 ]

Firefox uses pdf.js to render PDFs under the hood, I checked its latest version but didn't see any warnings/errors despite the garbled kerning. I spent about half an hour trying to figure out what's going on but unfortunately found nothing. I suggest you generate a test PDF and create an issue at https://github.com/mozilla/pdf.js There's plenty of people who actually understand how PDF rendering works (unlike me).

14866. Tom 7 (74.109.249.246) – 03 May 2024 09:44:54 [ "April" 2024 ]

And recordings: Supposedly someone did record the event, but I don't have the video (yet). The director's cut for this project will be the YouTube video, although I'm not opposed to completionists watching the live recording!

14865. Tom 7 (74.109.249.246) – 03 May 2024 09:42:17 [ THPS Rules! ]

jonas: Thanks for the bug reports :) I think all the cases of extra spaces are a known problem where certain constructs (often it's a comment) result in adjacent text nodes that don't get fused. It's surprisingly hard to fix well, since the right fix would be to finish the somewhat delicate hand-written bytecode that concatenates layout nodes. I just ran out of time to get that done before the deadline, although since the deadline I've rewritten that part of the compiler so that it's not as bad to hand-write bytecode, so I plan to fix this one soon. The spaces burn my eyes too!

Regarding the ones where space seems too narrow, sometimes this is apparently the actual metrics for Palatino italics when mixed with roman (but I agree it looks bad and should be tweaked). Other times it is a consequence of hacks (in the paper's code) to work around the problem (with the language implementation) described above. Most should be fixable. Of course I absolutely deserve to be taken to task on these things.

Fixed the typo! It's a miracle if that's the only one, since I didn't have any spell checking support!

Colorful graph: Yeah, this is described imprecisely. 00 actually means 1 thread, and 01 means 2 threads, and so on. I just looked and the data are correct, but I didn't do the +1 when generating the image. It's fixed in case I regenerate that.


Others: Thank you for your enthusiasm! There is a video coming. I have essentially all the content now and I am still alive.

14864. Tom 7 (74.109.249.246) – 03 May 2024 09:25:30 [ "April" 2024 ]

I generated the PDFs myself so there's possible I made a mistake, or possible that Firefox just doesn't support something I'm doing. :( I should try to fix this since I actually may use BoVeX for future papers. If anybody has any ideas (even like, an error message) I'm definitely interested!

Concentrating on the video for now though!

14863. Tim McCormack (130.44.147.140) – 02 May 2024 22:49:26 [ "April" 2024 ]

Yeah, at first I thought the aggressively bad kerning was part of the joke, but the example monospaced paragraphs are *really* messed up in both Firefox and also in whatever my default PDF viewer is in Debian.

I took a look at the SIGBOVIK PDF and it looks *way* better. (Page 131 intra-diegetic/135 extra-diegetic for others who want to read it there.)

14862. Anonymous (192.122.237.12) – 02 May 2024 11:29:39 [ "April" 2024 ]

What is this? A crossover episode?

It's great seeing y'all get together! Thanks for sharing!

14861. DigiNova (49.47.16.49) – 02 May 2024 05:57:22 [ "April" 2024 ]

noone can be a better speaker/candidate/chief guest of a "comedy" show titled "An Evening of Unnecessary Detail" orz
big fan of all three in ascending order of respect++ from left to right in the image

14860. Anonymous (129.2.194.82) – 01 May 2024 22:51:40 [ "April" 2024 ]

I'm not sure why, but Firefox on macOS absolutely chokes on the Badness 0 PDFs (both versions); the kerning is all over the place.