Comment on AI Artefacting

• areyouevenreal@lemm.ee ⁨2⁩ ⁨months⁩ ago

  LLMs have legitimate uses today even if they are currently somewhat limited. In the future they will have more legitimate and illegitimate uses. The capabilities of current LLMs are often oversold though, which leads to a lot of this resentment.

  source

  • leftzero@lemmynsfw.com ⁨2⁩ ⁨months⁩ ago

    LLMs have legitimate uses today

    No they don’t. The only thing they can be somewhat reliable for is autocomplete, and the slight improvement in quality doesn’t compensate the massive increase in costs.

    In the future they will have more legitimate and illegitimate uses

    No. Thanks to LLM peddlers being excessively greedy and saturating the internet with LLM generated garbage newly trained models will be poisoned and only get worse with every iteration.

    The capabilities of current LLMs are often oversold

    LLMs have only one capability: to produce the most statistically likely token after a given chain of tokens, according to their model.

    Future LLMs will still only have this capability, but since their models will have been trained on LLM generated garbage their results will quickly diverge from anything even remotely intelligible.

    source

    • areyouevenreal@lemm.ee ⁨2⁩ ⁨months⁩ ago

      This is false. Anyone who has used these tools for long enough can tell you this is false.

      LLMs have been used to write computer code, craft malware, and even semi-independently hack systems with the support of other pieces of software. They can even grade student’s work and give feedback, but it’s unclear how accurate this will be. As someone who actually researches the use of both LLMs and other forms of AI you are severely underestimating their current capabilities, never mind what they can do in the future.

      I also don’t know where you came to the conclusion that hardware performance is an issue, given that LLM model size varies immensely as does the performance requirements. There are LLMs that can run and run well on an average laptop or even smartphone. It honestly makes me think you have never heard of LLaMa models inc. TinyLLaMa or similar projects.

      Future LLMs will still only have this capability, but since their models will have been trained on LLM generated garbage their results will quickly diverge from anything even remotely intelligible.

      You can filter data you get from the internet to websites archived before LLMs were even invented as a concept. This is trivial to do for some data sets as well. Some data sets used for this training have already been created without LLM output (think about how the first LLM was trained).

      Sources:

      • arxiv.org/abs/2402.06664
      • doi.org/10.1145/3607505.3607513
      • medium.com/…/ml-story-mobilellama3-run-llama3-loc…
      • github.com/jzhang38/TinyLlama
      • Both my own personal direct experience and the experience of my research supervisor, with a dash of basic computer systems knowledge

      source