Comment on Migrating images to object storage, images will be inaccessible for a bit

Blaze@reddthat.com ⁨5⁩ ⁨months⁩ ago

• Sal@mander.xyz ⁨5⁩ ⁨months⁩ ago

  Thanks!! I faced an unexpected problem, but now the files are being transferred ^_^

  The transfer is quite slow. I have timed it for a bit and if the “Used Space” data on the object storage dashboard is correct, files are being transferred at about 120 MB / minute… If that rate is the average, this will take about 40 hours :/ But hopefully the object storage dashboard is lagging. We will see.

  source

  • Blaze@reddthat.com ⁨5⁩ ⁨months⁩ ago

    Fingers crossed 🤞

    source
  • Shdwdrgn@mander.xyz ⁨5⁩ ⁨months⁩ ago

    Wow that’s depressingly slow! My previous file server would transfer at around 75MB/s (not minute) and I thought that was pretty bad. Hopefully you can figure out where the bottleneck is at.

    source

    • Sal@mander.xyz ⁨5⁩ ⁨months⁩ ago

      I thought that worse case scenario this would be an overnight transfer, but only 40 GB have been transferred so far.

      The server’s bandwidth use is well below the limit, so I think that the issue might be on the receiving end (Contabo’s S3-compatible object storage).

      I had to decrease the number of parallel connections because otherwise I would get too many errors and the transfer would stop, so it might not be the transfer speed of a single file, but rather the amount of time the server has to wait between initiating transfers. I am not sure, but at ~50GB per 12 hours this will end up taking almost 4 days… I will ask in the Matrix chat to see if this was also the experience of others who migrated.

      source