This past June, I put together a write-up about two major approaches to backfilling conversations. The ability to properly backfill conversations means we will be able to make major inroads toward solving the feeling that the fediverse is quiet.

I, alongside several other members of the SWICG Forums and Threaded Discussions Task Force (ForumWG) have been working toward building implementor support for Conversational Contexts — the ability to explicitly classify a set of objects as belonging to a conversation, whether that be a topic, reply tree, or similar.

I am happy to report that we have made some wonderful inroads this past few months!

jesseplusplus@mastodon.social has been working closely with the Mastodon team to allow software to backfill from Mastodon — this pull request has now been approved!

I have been working with both nutomic@lemmy.ml and rimu@piefed.social to allow software to backfill from Lemmy and Piefed, respectively. (Lemmy PR, Piefed issue)

This marks a major milestone in the adoption of conversational contexts. With Mastodon on board backfill will be possible with the majority of the microblogiverse. With Lemmy and Piefed on board, backfill will be possible with the majority of the threadiverse.

Remember that pfefferle@mastodon.social was an early adopter of conversational contexts, and we have been able to backfill from WordPress blogs for quite awhile now (so that's the blogiverse too)

I for one, am eagerly awaiting the next version of all of these softwares!!

top 26 comments

sorted by: hot top controversial new old

[–] julian@activitypub.space 17 points 14 hours ago (1 children)

Thanks for the link back, I appreciate it!

One neat side effect of this is that if the microblogiverse backfills from the threadiverse, and vice versa, then you have in effect built a bridge connecting the two disparate 'verses.

We're not there yet — this step only exposes conversational contexts so that consumers (like NodeBB) can backfill from them. Neither Lemmy nor Piefed have signalled intent to consume contexts yet, but as it is a relatively new extension, there is no telling what may happen in a week, a month, a year... :)

Also rimu@piefed.social is pretty darn responsive and if you as (very) nicely that would go a long way.

[–] wakest@piefed.social 5 points 14 hours ago

Thanks for the context context!

[–] Goodlucksil@lemmy.dbzer0.com 33 points 17 hours ago (2 children)

Can anyone ELI15 what backfilling conversations means?

[–] wakest@piefed.social 46 points 16 hours ago (2 children)

Sometimes you see a post on your home instance and it looks like it has no replies but then you go look at it on the remote view and there are lots of replies. Backfilling is like making sure if you see a post it has all the comments

[–] Psythik@lemmy.world 2 points 9 hours ago* (last edited 9 hours ago) (2 children)

What's "the remote view"? Never seen that feature in any Lemmy app before. But if it means more comments on Lemmy, I'm all for it.

[–] Natanael@infosec.pub 1 points 4 hours ago

The post you replied to comes from a different instance than your own, so does my answer. When you're logging into your instance, the view of their and mine posts are both remote to you.

Sometimes in Mastodon you'll only see the specific post that you're opening a link to directly, not other posts before or after. This tries to fix that.

[–] pwalker@discuss.tchncs.de 1 points 4 hours ago* (last edited 4 hours ago)

Well you have your "home" lemmy instance which you are logged in to. Everything you see goes through your home instance. But many things you will see originate on a completely different "remote" instance. Your home instamce nees to be federated and synced in order to the everything that happens in the remote/origin instance

[–] AwesomeLowlander@sh.itjust.works 8 points 16 hours ago (3 children)

I don't use mastodon, but is that an issue for the Threadiverse? I thought all replies are collected by the host instance of the community.

[–] julian@activitypub.space 12 points 14 hours ago* (last edited 11 hours ago)

> is that an issue for the Threadiverse? I thought all replies are collected by the host instance of the community.

You're not wrong about that. Threadiverse software (Lemmy, Piefed, and NodeBB too) use something called 1b12 to synchronize content between instances.

If you follow a community, you set up the synchronization from your server to that community from that point forwards.

This is similar, but not quite backfill, which is what happens if you follow a community mid-way and don't have any of that stuff from before.

Also you can't 1b12 synchronize with Mastodon, so that unfortunately is a dead-end, but you would hopefully be able to backfill from Mastodon.

[–] dfyx@lemmy.helios42.de 17 points 16 hours ago (1 children)

It may still be missing stuff from before the first person from your home instance joined the community which can make younger communities feel empty.

[–] AwesomeLowlander@sh.itjust.works 8 points 16 hours ago (2 children)

Ah ok. That's the problem that the major instances solved by having bots auto join all new communities in each other, right?

[–] FundMECFS@anarchist.nexus 3 points 10 hours ago

yep. https://lemmy-federate.com/ lets you do this.

[–] curbstickle@anarchist.nexus 5 points 16 hours ago* (last edited 15 hours ago)

Yup, this removes that need.

Edited to add: and more BTW, including addressing brief network outages messing with sync and other issues, depending on the server type/activitypub implementation.

[–] wakest@piefed.social 3 points 16 hours ago (1 children)

everything is weaved together! What happenes when a mastodon account responds to a piefed account and then that gets shared by a Friendica account etc.

[–] AwesomeLowlander@sh.itjust.works 2 points 16 hours ago (1 children)

I have no idea :)

[–] wakest@piefed.social 4 points 16 hours ago (1 children)

My point is that there is never a hard line between any other parts of the fediverse so if there is a new feature or way one part does something it will always have an effect on the other parts.

[–] wakest@piefed.social 2 points 16 hours ago

Wow that is a badly written sentence! I think you get my point tho

[–] Rentlar@lemmy.ca 9 points 16 hours ago

AFAIK, It's to solve or mitigate the "Replies from other servers may be missing" issue.

Essentially imagine you are signed in on server A, responding to or looking at a comment from server B. But people on servers C, D and E have faved, boosted or replied to that same comment. Unless you or someone on your server had followed people on the other servers, you can't see those comments or their contributions to the boost count, unless you go to view the comment on server B's site.

Backfilling means server A fetching those other actions from other servers somehow, so that they will show up when you view it from your own server reliably. Examples of that somehow could be, obtaining all the info from server B (localized single source of truth), it could be collected individually from other servers, from a centralized server, or other means.

[–] MxRemy@piefed.social 9 points 17 hours ago

Whoaa that's super exciting!!

[–] ozoned@piefed.social 3 points 15 hours ago

Julian is an awesome guy! I just spoke with him on the latest episode of Fireside Fedi if anyone is interested: https://video.firesidefedi.live/w/q3z3C6JrpitEUxwFmwiTUo

[–] some_kind_of_guy@lemmy.world 3 points 15 hours ago* (last edited 15 hours ago) (2 children)

For some reason I assumed this was already how the fediverse worked, but I haven't been here very long and it does explain some things, including the "empty" vibe in some lesser-populated places.

This is super exciting for the fediverse and, naturally, I have questions. While this change will mostly bring positives and a better experience for users, there could also be more opportunities for shenanigans.

What considerations are being given to data integrity/mutability and trust? Will all servers that touch a post have a distributed record of all comments and give network confirmation (a la blockchain)? Or does one server (e.g. the originator of each post, or the server with the most resources) act as a single authority of that post? Something else?

Could one server be instructed to "go rogue" and submit bad content to the network, or go on a deletion/overwriting spree that ends up becoming permanent?

What about resources? What impact will backfilling have on your average dude hosting a small instance?

This is just where my mind goes, you see. I'm sure all this and more have been discussed and figured out already. If a public discussion is available to look at, I would love a link!

[–] julian@activitypub.space 3 points 14 hours ago (1 children)

Those are all very good questions, and exactly the sort of things that would be discussed at the ForumWG.

Backfill is just one of the things (the main thing, currently) we touch on, but one of the more important ones, because the potential to ensure you have the entire conversation is important from a data completeness standpoint.

The thing to remember is that there's no one "owner" of a conversation. Right now it's a pretty loose association... individual posts and notes can declare that they are part of a context, even if that isn't the case. This beats the current system where there is no association at all.

The difference here is that as a consumer of backfill, I can actually go to the context and verify this. We can extend this later on with context ownership, and defer responsibilities like moderation, interaction policies, forking/merging, etc.

The long view of this is we intend to increasingly solidify the association between context and object over time.

[–] some_kind_of_guy@lemmy.world 1 points 8 hours ago

Very cool, thanks for your response!

[–] wakest@piefed.social 3 points 15 hours ago (1 children)

There have been monthly meetings about this issue and surrounding ones that are open to the public if you really want to know more. There are years worth of discussions about it too but they are very spread around

[–] some_kind_of_guy@lemmy.world 1 points 8 hours ago

Thanks!

[–] lena@gregtech.eu 3 points 16 hours ago

So cool