During the 2020 lockdowns I discovered the Russian-speaking Indieweb community and we did an online HWC meeting that was my first (and only, so far) “in-person” IndieWeb community experience. At the moment, I’m having a very extensive “life happens” period but I’m very much looking forward to more community interaction once I have more time on my hands.

1. Comments sections, or
2. Twitter?

Especially when those two things can be more elegantly done (even in a distributed manner) with less flexible solutions.

What are other fun/weird things you could do with webmentions?

His point on Javascript I'm not entirely in agreement with. For one, webmention.io is fully self-hostable so having a static site with webmention content provided at runtime isn't at all bound to someone's ability to provide a service for free (I use webmentiond for this). For another, this sounds like something a caching layer was designed to solve if anyone's site hits Scale. Add to this that runtime fetching of other people's PII is arguably superior, as this gets into a little. And finally, I don't think that displayed webmentions are something that necessarily should be machine-parsed. My content is marked up with mf2 because I want to make it easy for people to parse and consume my content. I don't own the content I'm displaying as replies; even if there's an etiquette about excerpting for comments-section-type-usage, why should I be exposing it for even-more-indirect consumption?

Nog enkele bedenkingen …

De meeste site-eigenaren willen natuurlijk dat er naar hun pagina’s wordt gelinkt. Denk aan SEO en zo. (Nu, iedereen wil op sociale media viraal gaan, ook. Tot een domme tweet je carrière dreigt te nekken. Dan wil je terecht ‘vergeten worden’.)

En het insluiten van ‘Open Graph’-inhoud is ook geen issue. Dat díént daarvoor. Als Facebook, en die cachen ook alles, dat mag, dan een ander ook. Waarom zou dat voor h-card-gegevens anders (moeten) zijn? (Omdat dat altíjd persoonlijke gegevens zijn?)

Voor de volledigheid: het originele artikel ging in de eerste plaats over het insluiten van veranderlijke inhoud, meen ik, en pas daarna over eventuele juridische aspecten.

Evgeny Kuznetsov

Owning your own data is great. I’ve been using this website as the central IndieWeb point of my online life for over five years, and I love it. However, the joy of owning your own website comes bundled with great responsibility: as the website owner, I am responsible for what’s on my site and fo…

• Mastodon icon

Owning your own data is great. I’ve been using this website as the central IndieWeb point of my online life for over five years, and I love it. However, the joy of owning your own website comes bundled with great responsibility: as the website owner, I am responsible for what’s on my site and fo…

Dit vind ik een interessant experiment. Hoe kun je een link naar een andere site rijker maken, zonder extra plugins, zelf data opslaan, maar door gebruik te maken van de protocollen die op die andere site staan. Het is nog verre van een perfecte technische oplossing en het is geen plugin die je even aanzet. Maar het begin is weer gemaakt om verder te bouwen op de indieweb-bouwstenen. En dat is altijd goed nieuws.

https://diggingthedigital.com/leveraging-indieweb-to-avoid-storing-others-data/

A question that immediately arises when you start thinking about writing your own webmention endpoint: “How do I store?” You start having thoughts of databases, good and bad things about them, but with that way of thinking it’s much easier if you just self-host webmention.io (it’s open-source) and avoid all the hassle. My website is built with Hugo, and I really like the fact that everything is stored as simple text files in one git repository. It’s only logical to store webmentions the same way, isn’t it?

What pushed me was the article about doing webmentions on a Hugo-based website: why not make the server check webmention.io from time to time and save the new webmentions to a JSON, I could surely build on that. The idea of storing them in a separate file in /data/ was immediately discarded: Page Bundles were invented so that we could store each post (index.md) together with the webmentions (webmentions.json) in the same subdirectory. The webmentions that target the site as a whole (such as person mentions) I can store in a webmentions.json right in /content/ (and not yet display)…

The author of the aforementioned article does the saving in JS and runs it on Netlify. I don’t do Netlify, so I need a separate tool for saving, which fortunately has already been developed. All I needed was to add an option to save to separate subdirectories, which I did. I also added more options while I was at it (not sure if anyone needs those).1 Also, the code that does the saving to subdirectories will likely be reused later when I write a webmention endpoint.

Here comes the first “oops”: many of the site’s pages (including almost all the Reactions) are text-only, so I was too lazy to make Bundles for them. In Hugo, the contents of a page addressed as https://evgenykuznetsov.org/posts/2021/old-comments/ can reside either in /content/posts/2021/old-comments.md, or in /content/posts/2021/old-comments/index.md; in the latter case the same subdirectory can be used to store the images and whatever is needed on the page, and this is exactly what’s called a “Page Bundle”. Since I was going to store webmentions in Bundles, the first way of storing the pages doesn’t really work, and I have already accumulated enough pages stored in this manner. I needed to re-save all of them in another way, each in its own subdirectory, and doing it manually would take all summer, especially considering different language variants. Bash to the rescue:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/bin/bash
set -e

wd=$(pwd)

if [[ "$1" != "" ]]; then
        wd="$1"
fi

for x in "$wd"/*.md; do
        case "$x" in 
                *.ru.md)    suffix="ru.md";;
                *.en.md)    suffix="en.md";;
                *.md)       suffix="md";;
                *)          continue;;
        esac
        name="${x%%.*}"
        if [ ! -d "$name" ]; then
                mkdir "$name"
        fi
        mv "$x" "$name/index.$suffix"
done

…and long live Stack Overflow!

From there it was easier. I removed webmention.js from the templates, wrote the processing of the bundled JSON (which in many ways was the same webmention.js translated from JavaScript into Hugo templating language and customized to my needs), fixed CSS (I now know much more about CSS/SaSS than I did in June), and wrote a template for comments RSS-feeds. I also marked avatars and names for dynamic loading and uncovered a couple of bugs in indieweb-glue along the way: some things were not cached when they should have, especially the 404s that also lacked CORS headers, I fixed all that. Everything works! Admittedly, webmentions no longer appear immediately (only after the next rebuild of the site), but now I can pre-moderate them if I ever feel like it.

Yay, now I could also display older webmentions, the ones I received back when I was running Known; I did save them when I moved from Known to Hugo, and I used the same JSON structure as webmention.io! I only wonder what happened to all the avatars… Oh, it turns out that Known stored the avatars for the received webmentions, and all the links are to evgenykuznetsov.org/service/web/imageproxy/ that no longer exists. Those links need to be removed. Not by hand, of course, we have jq and the magic of find, thank Discordia for Stack Overflow!

find ./ -type f -name "webmentions.json" \
    -printf \
        "jq -c 'del \
            (..|.photo?| \
            select(contains \
            (\"evgenykuznetsov.org/service/web/imageproxy\")?) \
            )' %p | \
        sponge %p\n" \
    | \
sh

Hmm, what was the format I saved the comments from Google+, LiveJournal and diary.ru in? The same webmention.io-inspired JSON, how cute! Let’s display those, too… Oops! Likes in G+ had no date so the RSS template breaks: Hugo happily parses a date from nil and gets a value of error.Error type that can’t be further processed. Another check there, and here we go: right here on my website, the comments that are over 15 years old! So fascinating to read, especially the comments from back when the blog was much more personal… Some nicknames I can no longer remember, some have departed from this world, some I still talk to regularly, and some provoked an urge to immediately contact. A blog that is almost a half of the life old, and with comments, beats any personal journal.

Writing my own webmention endpoint is next on the roadmap.

1. By the way, if you do the same and save webmentions as JSON, I strongly advise pretty-printing the file (the same way jq does by default). This takes hundreds of extra bytes per page but makes a tremendous difference in git diff. ↩︎

However, paying for Heroku isn’t even an option: I am Russian, I live in Russia, so paying online for non-domestic services requires jumping through too many hoops these days.

Of course, indieweb-glue will live on. I’ll be hosting it on the same VPS I host this website on, at least for the time being. However, the indieweb-glue.heroku.com URL will have to go, so if you are using indieweb-glue1 (and don’t host your own copy), be sure to update the URL to indieweb-glue.evgenykuznetsov.org before the end of November.

I should have used my own domain for the URL from the start, but learning how to set it all up seemed too much of a hassle back then2. Well, here’s a lesson: own your data and own your URLs, IndieWeb is good for you!

1. I know at least  Zack does, or did at some point. ↩︎

2. It turned out to be extremely simple. ↩︎

The final push was reading about the visible links idea which makes a lot of sense to me. As that article points out, Wikipedia does hovercards for a reason, and while I don’t claim my writing is similarly important, the general idea of giving a reader more context with less effort is rather appealing.

One thing that has been holding me from attempting to implement this feature earlier is mobile. Hovercards don’t make sense with touch-screen interfaces and I was reluctant to add to my website something that would be unusable on mobile. Not that I’m a huge mobile fan, but I try to make sure my site is available and accessible on mobile, too. On the other hand, the gitweb interface is hardly accessible on mobile, either, but nor does it make real sense to access it on mobile, so I have it on my site. Wikipedia doesn’t do hovercards on mobile either; what works for them is good enough for me, I suppose.

With these thoughts, I went on to get my hands dirty with the technical side of things. It was very nice of  capjamesg to have recently sorted out all the hard stuff (I know, I know, I need to teach myself JavaScript for real), so I didn’t have to start from scratch. Of course, I don’t use the backend James uses, so that’s what I needed to sort out on my own.

Where should the information for a hovercard come from? The most straightforward approach is to use the information that the linked page provides about itself. For the reasons I laid out earlier, I didn’t want to pre-fetch and store that information the way James does, so the obvious solution was to leverage indieweb-glue. Of course, it had to be taught to return the information about pages in addition to people. One form for a page to provide such information about itself is the Open Graph protocol, it is rather widely used, so that’s what I decided to start with.

After some mild code refactoring, I added Open Graph meta tags processing to indieweb-glue and modified James’s script to fetch the information from there. I also made it look for a specified class instead of an <article> tag since it gives me more control over which pages and parts of the pages hovercards appear on. With some CSS thrown into the mix, it’s all working quite to my satisfaction now. Surprisingly so, I must say.

As an added benefit, the same script can be used to produce hovercards for footnotes1 on the fly, and that’s something I’m even more fond of. Footnotes have been my pet peeve for a long time. They are a really nice instrument to have when writing a text, allowing one to avoid interrupting the flow of thought with distractions and digressions. However, in digital media, footnotes are often hidden beyond the screen edge, and scrolling or following a link to look at them is usually too much distraction. On paper, all you need to do is look down, and we are good enough at returning our sight focus back to where it was on the page so that taking a look at a footnote doesn’t interrupt our reading process too much. Having footnotes in hovercards feels like a reasonable compromise.

The result is not yet perfect, and all this is still a work in progress. CSS needs more figuring out: my hovercards don’t always look good, especially when the image is in portrait orientation. Also, indieweb-glue could (and should) use things beyond the Open Graph protocol to extract useful info from web pages. A good example is Wikipedia: its pages lack og:description, but the first paragraph of the text itself looks like a perfect thing to have in a hovercard linking to a Wikipedia page. I also plan to modify the script so that it recognizes some special cases, such as person mentions, where populating the hovercard with H-Card data is appropriate, for example.

Of course, I’d highly appreciate any comments, advice or thoughts on how to make these things better. For a work in progress, these are always welcome.

As I was writing this post, I noticed that the first link (the one to  Maggie Appleton’s article) produces a hovercard with a nice cover image that is obviously autogenerated. Of course, I immediately wanted something like this for my posts that lack a cover image.

Technically, the Open Graph protocol lists og:image as one of the required properties, meaning that every page should have one. Of course, finding a good (or even acceptable) cover image for every post can be too much hassle. There are three possible ways to deal with it: have a generic site-wide image for all the pages lacking their own images, have the images autogenerated, or have no image at all for some pages. The latter option violates the protocol, obviously, but it is used widely enough, and I also resorted to it until now.

I asked around (in the Russian IndieWeb chatroom, naturally). The way most people (including Maggie) seem to go about it is by using Playwright, a headless browser developed for testing, to open the page and make a “screenshot”. This is a fascinating hack, but it doesn’t quite suit my workflow. I’ve also found a piece of software tailored specifically for Hugo, as well as some more general guidelines on how to generate cover images programmatically with JavaScript or Python. Those guidelines got me really thinking, and I ended up writing a Hugo template that generates a picture automatically for every page that doesn’t have a featured image.

It works, although it, too, is very much a work in progress. Image processing in Hugo is somewhat limited, and also I’m not a very visual person, so these things are always hard for me. It’s currently a very simple script that only uses the site’s favicon, the author information and the page title, and it does nothing to adapt to the title’s length (so the text would sometimes overflow or look out of place). The good thing is, most of it can be fixed with time, and the script expanded to take into account more information about the page. Still, even as it is, it’s better than nothing, I think. At least every page does have og:image property now.

1. I know I tend to abuse this feature sometimes. ↩︎