Comment

Nothing4You@programming.dev ⁨5⁩ ⁨months⁩ ago

I can’t tell you why you’re lagging but you’re clearly lagging quite a bit behind.

which country is your instance located in?
did you (or someone else on your instance) recently subscribe to a bunch of high traffic communities on lemmy.world, which would make lemmy.world send more activities to you?

lemmy by default only sends activities in a community to another instance if there’s at least one subscriber to the community on that instance. if you’re located far from finland, where lemmy.world is located, you might have been able to keep up just enough before this, although this isn’t the first time as the graphs above show.

source

Sort:hotnew top

CanadaPlus@lemmy.sdf.org ⁨4⁩ ⁨weeks⁩ ago
Hey, sorry for the late reply, but I’m trying to figure out who runs this graph, because if it needs something like a bit of funding to store data for longer periods I’d like to help. Do you know, or know where you heard about it?

source
- Nothing4You@programming.dev ⁨4⁩ ⁨weeks⁩ ago
  Hi, I run this.
  
  What benefit do you expect from longer retention periods and how much time did you have in mind?
  
  The way data is currently collected and stored keeps the same granularity for the entire time period, which currently uses around 60 GiB for a month of retention across all monitored instances.
  
  source
  - CanadaPlus@lemmy.sdf.org ⁨4⁩ ⁨weeks⁩ ago
    Mainly, I’m interested in how Lemmy is growing and changing as a whole. If there was a way to store activity just weekly or even monthly that would help for what I’m interested in.
    
    source
    Nothing4You@programming.dev ⁨3⁩ ⁨weeks⁩ ago
    so all you’re looking for is the amount of activities generated per instance?
    
    that is only a small subset of the data currently collected, most of the storage use currently comes from collecting information in relation to other instances.
    
    source
    -> View More Comments
originalucifer@lemmy.world ⁨5⁩ ⁨months⁩ ago
its running on AWS, US. you can see on 4/24 i finally solved for a long running bottleneck (disk writes), and it was smooth sailing until recently.

im not sure if anyone else has subscribed to any new communities, but ive been subbed to most of the big ones for awhile.

i don’t see any saturation in the server network activity, connections, etc. my queues are not overloaded, and as i mentioned other instances are very performative.

maybe i just need to be patient and let it catch up. those graphs are awesome

source
- Nothing4You@programming.dev ⁨5⁩ ⁨months⁩ ago
  lemmy currently only sends one activity per receiving instance at a time, so there is a round trip for every single post, comment, vote, etc., before the next activity will be sent. you wouldn’t see any increased number of connections, as there’s only a single one.
  
  do you have access logs for /inbox with the lemmy.world’s user agent? you might be able to derive some information from that if requests increased over time or something, maybe also response status codes?
  
  source