• sobchak@programming.dev
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 month ago

          HDDs have horrible random access times, so if you need to process or just copy a lot of small files, say photos, there’s a significant penalty.

            • sobchak@programming.dev
              link
              fedilink
              English
              arrow-up
              1
              ·
              edit-2
              1 month ago

              Rsync, syncthing, backups, mp3s, photos, json files; idk, a lot of tasks involve large amounts of small files. I personally ran into this problem training models on millions of photos. My GPUs would only get up to 25% utilization with mirrored HDDs, so I had to switch to SSDs.

              Edit: the difference is also significant when compiling large projects or just using git. I imagine some game servers need a lot of random accesses too.

              • Gladaed@feddit.org
                link
                fedilink
                English
                arrow-up
                1
                ·
                edit-2
                1 month ago

                Why are you doing that on a network storage as opposed to on device?

                Also who got millions of photos at home?

                • sobchak@programming.dev
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  1 month ago

                  Not enough room in the GPU machine for all the HDDs I needed.

                  Also who got millions of photos at home?

                  People working on biological datasets.

                  • Gladaed@feddit.org
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    ·
                    edit-2
                    1 month ago

                    Why are you doing that recreationally? How are you different from a researcher?

                    Why are you using a home server for that?