No matter what I do I don't see any posts from .world. I checked and mander does not seem to block .world (kinda why I liked this instance) and even weirder is how my feed gets filled with mostly .ml posts with almost no activity no matter how I sort.

This did not happen before and got me thinking, I don't even know how this works across federated instances.

Does anyone know:

How does the sorting work on an instance? Is it all the same no matter what instance it is?
Did anything change in the last few months, that would cause all .world (and I assume others) posts to not show?
Other then changing instances what can I do to mitigate the weird slanted results?

top 47 comments

sorted by: hot top controversial new old

[–] j4k3@lemmy.world 31 points 6 months ago* (last edited 6 months ago) (2 children)

The Lemmy algorithm:

https://join-lemmy.org/docs/contributors/07-ranking-algo.html

The instances communicate between each other using special bots in the background that transport info between instances. Dot world is too big and makes too many requests to other instances. Most instances are reducing the number of transport bots back to dot world right now. There is supposed to be a fix in the next Lemmy version, but the code base is somewhat slow moving due to only two devs and it is written in Rust. Rust is a hard language like C, and not too many here are able to contribute to it, even though it is like the new gold standard of code.

So you might see a delay between posting and replies or the interaction may come in bursts that correspond with the transport bots carrying content between instances as the host admin have configured their instance.

[–] gravitas_deficiency@sh.itjust.works 12 points 6 months ago (3 children)

I’m actually mildly surprised it’s still only two devs. Are they treating it like a walled garden, or is there really a complete lack of interest in contributing to the codebase?

[–] j4k3@lemmy.world 6 points 6 months ago (1 children)

Not sure. There was some controversy with some of the devs making alt front ends and admins complaining about the slowness. I've seen mention of one of the two devs learning Rust just to participate. So it is not entirely a walled garden. The front end devs wanted to make an alt from scratch but in something like JavaScript although I don't recall the details exactly. There were a lot of red flags related to privacy and understanding the community at large in the posts I saw from them. When asked why they weren't adding pull requests with Rust in order to address their complaints I got no reply.

All that said, I'm no dev. I can read in to around half the code I come across if I really try, and can successfully modify maybe half of that if I spend a few days on it, but I suck at clever code and the DRY cult types. I haven't tried to look into Lemmy in any depth beyond figuring out the basics.

[–] M0oP0o@lemmy.world 5 points 6 months ago (1 children)

Hmmmm, and it does not help the controversy when this issue seems to happen to then fill smaller instances with .ml posts.......

[–] j4k3@lemmy.world 3 points 6 months ago (1 children)

The .ml instance is run by the developers. If it isn't well configured we would have no reason to be here.

[–] M0oP0o@lemmy.world 2 points 6 months ago (1 children)

Well we would also not want to be here if .ml was favoured over any other instance, that would kinda make the idea of federation a bit pointless.

[–] j4k3@lemmy.world 2 points 6 months ago

It follows the first to market principal in many of the most active communities, and it is the most federated instance. Many instances that are not federated with each other are federated with .ml. You still won't see those comments between instances. Like from my main account here on .world, I can't see hexibear or beehaw stuff, but from my .ml account I can see them. I have accounts on many instances in order to help federate new communities and to check biases/instance behaviors.

I came over a few days before the rexodus and subscribed to the active communities before the influx. That sub list is still centered around the most active communities, and the majority of those are from .ml and before I joined Lemmy.

Instances all have different flavors. I don't like using my .ml account as a main. I've tried it. But I find they are the center of the most interesting and productive conversations for a more broad audience, while Beehaw has the most positive and friendly conversations overall. The main benefit to .world is the speed of connectivity, general audience scope, but with a strong anti asshat policy.

[–] nutomic@lemmy.ml 6 points 6 months ago

We are two fulltime developers and a handful of devs who regularly contribute in their free time. We could definitely use more devs but the donations are simply not enough.

[–] Blaze@reddthat.com 0 points 6 months ago (1 children)

Rust is a niche language

It's like starting a book club in Esperanto compared to English. Sure, Esperanto is supposedly a better language, but very few people know about it

[–] gravitas_deficiency@sh.itjust.works 0 points 6 months ago (2 children)

Rust is not a niche language. It’s a strict and strongly opinionated language by design. People with background in strongly typed languages, who additionally use opinionated linters and formatters have an easier time adjusting. JavaScript “devs” (note: distinct from “software engineers”) probably pull their hair out over a lot of stuff in because in my experience, many js devs know enough about the language to work proficiently in a couple of frameworks, but haven’t really dug into the nuances of the language, and also have limited experience with strong typing.

[–] Carighan@lemmy.world 4 points 6 months ago (1 children)

It’s a strict and strongly opinionated language by design.

So it's a nice language.

[–] SorteKanin@feddit.dk 1 points 6 months ago

Very nice even :D

[–] Blaze@reddthat.com 2 points 6 months ago* (last edited 6 months ago) (2 children)

My main argument about Rust being a niche language is how few contributors there are to the Lemmy codebase.

Mbin (PHP) has more spread across its contributors:

But maybe Rust isn't that niche, but the Fediverse apps and projects are niche themselves.

[–] nutomic@lemmy.ml 4 points 6 months ago

The Lemmy frontend is written in Typescript which is a very popular language, yet it has even less contributors than the backend.

[–] SorteKanin@feddit.dk 1 points 6 months ago (1 children)

But maybe Rust isn’t that niche, but the Fediverse apps and projects are niche themselves.

Lemmy is niche even within the fediverse, where microblogging still dominates and the threadiverse style apps are smaller. It's just not a very large space.

[–] Blaze@reddthat.com 2 points 6 months ago (1 children)

I guess Reddit and similar link aggregators are just much smaller than we all think they are.

[–] SorteKanin@feddit.dk 1 points 6 months ago

Even reddit is still niche when it comes to social media and has always been. It's become a little more mainstream the last few years, but for most people social media still equals Facebook, Instagram, Twitter, Snapchat and such.

[–] muntedcrocodile@lemm.ee 1 points 6 months ago (1 children)

I believe there is some work being done on an alternative code base written in python with flask it has lots of promise and i recon with a bit of extra work it might allow for mastodon integration into one server which would be cool af.

Really want to get to the stage where an account is just a private key I hold and all federated services can work with a single username.

Imagine having a single username which is for all services.

[–] Blaze@reddthat.com 5 points 6 months ago

I believe there is some work being done on an alternative code base written in python with flask it has lots of promise and i recon with a bit of extra work it might allow for mastodon integration into one server which would be cool af.

https://piefed.social/ ?

Otherwise, for Mastodon integration, there is mbin with https://kbin.run/ for instance

[–] Blaze@reddthat.com 13 points 6 months ago (1 children)

Huge sync issues between Lemmy world and mander: https://grafana.lem.rocks/d/cdfzs0dwal3pca/federation-health-time-behind?orgId=1&var-instance=All&var-remote_instance=mander.xyz&var-remote_software=All&from=now-1h&to=now

I posted about it on !mander@mander.xyz a few days ago.

Another reason to not have all communities on LW

[–] M0oP0o@mander.xyz 6 points 6 months ago (2 children)

Thanks for the reply (I was worried no one would) but would that cause the weird behaviour I am seeing? The timing does not seem to match but it could make some sense.

If what you are saying is the cause then Lemmy as a federated whole may now be effectively split up due to sync issues?

[–] throws_lemy@lemmy.nz 5 points 6 months ago

Mander is not the only instance having this issue. Other instances like lemmy.nz (it was), reddthat also aussie.zone, monyet.cc have similar issue since months ago. I assume the culprit is cogent's de-peering drama not from lemmy itself.

https://www.thinkbroadband.com/news/9896-ntt-cogent-peering-dispute-increasing-latency-for-some-routes

https://www.reddit.com/r/networking/comments/1cu13bv/cogent_depeering_tata/

And those lemmy instances are hosted in APAC.

Also you can check federation issue with this

https://phiresky.github.io/lemmy-federation-state/site?domain=lemmy.world

And if I'm not mistaken our admin has solution regarding the issue a while ago, you may ask Dave.

[–] Blaze@reddthat.com 3 points 6 months ago (1 children)

Actually there are some comments from other people: https://reddthat.com/post/20632337?scrollToComments=true

Most of them are from LW, so they won't federate before a few days.

The issue is that if one instance has too many updates to send to another one (in this case LW to mander) it starts to create delays.

This mostly happens due to LW size, as you can see Reddthat has no issue

[–] M0oP0o@mander.xyz 3 points 6 months ago (2 children)

Oh wow, yeah its like night and day. Well now I guess I have to figure out a fix.

Sad times if I have to find another instance, again.

Any ideas what can be done longer term to address this (in general)?

Since I can't see anything here I will check on .world

[–] Cethin@lemmy.zip 1 points 6 months ago

If you're looking for a new home, I've enjoyed Lemmy.zip. I think it's on the smaller side, but I prefer that to everyone being on a few instances.

[–] Blaze@reddthat.com 1 points 6 months ago (1 children)

This bas to be adressed by the mander admins.

For instances, i would suggest lemm.ee or sh.itjust.works

[–] M0oP0o@lemmy.world 5 points 6 months ago

Maybe I should just crosspost this to !mander@mander.xyz

[–] Sal@mander.xyz 5 points 6 months ago* (last edited 6 months ago) (1 children)

Yes, sorry, there was some serious lagg in fetching posts from Lemmy World that persisted for several days and accumulated a 1-week delay.

But after upgrading Mander it is now fetching data from LW quite rapidly and it should be back in-sync in about a day and a half from now.

If you are curious about the ranking algorithm, there is some info here: https://join-lemmy.org/docs/contributors/07-ranking-algo.html

[–] M0oP0o@mander.xyz 3 points 6 months ago (1 children)

Thanks, I have been watching the graph of the lag behind time and its nice to see it now at 6.1 days.

[–] Sal@mander.xyz 3 points 6 months ago (1 children)

I find it satisfying to see the graph come down :)

[–] M0oP0o@mander.xyz 3 points 6 months ago (1 children)

Yeah, first time I have been cheering for line go down that I can think of.

[–] M0oP0o@lemmy.world 1 points 6 months ago

Almost under a day. I can now see .world, midwest.social, .ee and others in my feed.

[–] PixelAlchemist@lemmy.world 5 points 6 months ago

Should be the same across any instance.

Hot: calculated using the current score (up/down votes) in combination with how new it is.

https://github.com/LemmyNet/lemmy/blob/main/migrations/2023-10-23-184941_hot_rank_greatest_fix/up.sql

Active: calculated using current score in combination with the timestamp of the newest comment:

https://github.com/LemmyNet/lemmy/blob/main/crates/db_schema/src/aggregates/post_aggregates.rs

[–] Resol@lemmy.world 5 points 6 months ago

To this day I still have no idea what "hot" even means, even on places like Reddit, where I used to lurk before that place f***ing exploded.

[–] 9point6@lemmy.world 4 points 6 months ago (1 children)

Well, I can see your post on l.w, so it's not a total disconnect!

I wonder if there are any bugs in the new version of lemmy—do you know if mander has updated yet? The l.w admins tend to drag their heels updating because of some previously rough update experiences (I think mostly down to the size of the instance compared to others)

There were some issues with federation last time, perhaps it's something similar

[–] Blaze@reddthat.com 2 points 6 months ago (2 children)

The issue is from LW to mander, there is a huge delay

https://grafana.lem.rocks/d/cdfzs0dwal3pca/federation-health-time-behind?orgId=1&var-instance=All&var-remote_instance=mander.xyz&var-remote_software=All&from=now-1h&to=now

[–] 9point6@lemmy.world 3 points 6 months ago

Holy shit, a week delay is mad

[–] M0oP0o@lemmy.world 2 points 6 months ago

Well there is a big issue with LW to any smaller instances if this is working like you think. Right now being on mander is like having less then half of lemmy and I would guess that if it can happen with mander it can/is happening with others.

[–] M0oP0o@mander.xyz 3 points 6 months ago (1 children)

I decided to do a bit more then just "hot" and "active" and its the same across all of them except "controversial" and "Most comments".

"Most comments" has some .world posts in it but does not match using another instance. The only one that seems to line up is ironically sorting by "controversial",

This is really getting weird and is making it hard to use Lemmy for me. Like I like some of the posts listed but I can not believe the largest instance is not posting anything. To make me more confused this also started at the same time Mander got really slow, and to add to the weirdness refreshing the page gives entirely different sorting of posts maybe one out of five times.

[–] Blaze@reddthat.com 1 points 6 months ago (1 children)

Which one is your other instance?

[–] M0oP0o@mander.xyz 1 points 6 months ago (1 children)

Used to have .world but for this I just test without logging in.

[–] Blaze@reddthat.com 3 points 6 months ago

Mander has important delays with LW

https://grafana.lem.rocks/d/cdfzs0dwal3pca/federation-health-time-behind?orgId=1&var-instance=All&var-remote_instance=mander.xyz&var-remote_software=All&from=now-1h&to=now

[–] Aviandelight@lemmy.world 2 points 6 months ago (1 children)

I've been noticing this behavior on Mander too. I've been using my alt world account for now. I really hope the next update fixes things.

[–] andrew_s@piefed.social 4 points 6 months ago (2 children)

Please use your mander account to inform OP that their instance is wildly behind .world - see here

The delay and eventual deletion of comments and votes will have an obvious effect on mander's "hot" and "active" algos.

If you look at this post on mander, it's not got any of the comments, so there's no point anyone using a .world community to answer OP's question.

[–] Aviandelight@lemmy.world 2 points 6 months ago

I would but I'm on mobile and I can barely get it to load in the Connect app.

[–] M0oP0o@lemmy.world 2 points 6 months ago (1 children)

Yeap, no point in using mander right now ether.

[–] Aviandelight@lemmy.world 3 points 6 months ago

Okay I was able to get into the general communication community and post the question there.