Comment

Comment on New AI model can hallucinate a game of 1993’s Doom in real time

Even_Adder@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago

I mean, you’ve never seen a purple elephant with a tennis racket. None of that exists in the data set since elephants are neither purple nor tennis players. Exposure to all the individual elements allows for generation of concepts outside the existing data, even though they don’t exit in reality or in the data set.

source

Sort:hotnew top

MentalEdge@sopuli.xyz ⁨2⁩ ⁨months⁩ ago
Ok.

Try to get an image generator to create an image of a tennis racket, with all racket-like objects removed from the training data.

Explain the concept to it with words alone.

That’s the problem you’ll run into the second you want to realize a new game genre.

source
- Even_Adder@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago
  There are more forms of guidance than just raw words. Just off the top of my head, there’s inpainting, outpainting, controlnets, prompt editing, and embeddings. The researchers who pulled this off definitely didn’t do it with text prompts.
  
  source
  - MentalEdge@sopuli.xyz ⁨2⁩ ⁨months⁩ ago
    Obviously.
    
    But at what point does that guidance just become the dataset you removed?
    
    source
    Even_Adder@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago
    
    But at what point does that guidance just become the dataset you removed from the training data?
    
    The whole point is that it didn’t know the concepts beforehand, and no it doesn’t become the dataset. Observations made of the training data are added to the model’s weights after training, the dataset is never relevant again as the model’s weights are locked in.
    
    To get it to run Doom, they used Doom.
    
    To realize a new genre, you’ll “just” have to make that game the old fashion way, first.
    
    Or you could train a more general model. These things happen in steps, research is a process.
    
    source
    -> View More Comments