How to Create Art for a Book?

Submitted ⁨⁨5⁩ ⁨days⁩ ago⁩ by ⁨lefthandeddude@lemmy.dbzer0.com⁩ to ⁨stable_diffusion@lemmy.dbzer0.com⁩

I’m interested in creating art for a book, almost like art that goes alongside the text.

One of the difficult parts of this is that I want characters in the book to have a stable look and not change from image to image.

Is there a way to do this? I have experimented with different localized models and often there were artifacts and I couldn’t get consistent results. I am mildly intelligent with running local models, but I am neither an expert nor a computer genius.

It’s been about a year since I last tried anything and since then technology has progressed. If I can’t get the characters to look consistent from picture to picture, I’d rather not have images, as I can’t afford an illustrator.

source

Comments

Sort:hotnew top

Stampela@startrek.website ⁨5⁩ ⁨days⁩ ago
Qwen edit can take an image as a sample, and work with that. “The character in a victorious pose” would get whatever character you have, and reproduce it in a victorious pose. Couple of examples:

Image

Image

And a little janky because it IS generative AI after all…

Image

source
- lefthandeddude@lemmy.dbzer0.com ⁨5⁩ ⁨hours⁩ ago
  Is there a way to run this in a Linux-based system?
  
  source
  - Even_Adder@lemmy.dbzer0.com ⁨4⁩ ⁨hours⁩ ago
    Yeah, you can load Qwen Edit with Comfy UI. This tutorial for Comfy UI should help get you started.
    
    source
NihilsineNefas@slrpnk.net ⁨4⁩ ⁨days⁩ ago
The answer is you need to hire an artist.

If you can’t afford an artist, then don’t use the machine that steals art to cheat.

People will know, people will see, and people will judge your work with that as a demonstration of your lack of effort and morals.

source
hendrik@palaver.p3x.de ⁨5⁩ ⁨days⁩ ago
As far as I know, one way to do it is to use an image2image model / image editing model. You’d need to generate one reference image of your character. And then feed the reference image into it and tell it to draw that character in a different scenario. I haven’t done it in a while, not sure what people use these days to do it locally.

source
- lefthandeddude@lemmy.dbzer0.com ⁨5⁩ ⁨days⁩ ago
  That sounds so hard to do locally. I don’t even know where I would begin. Is that expert level stable diffusion?
  
  source
  - Even_Adder@lemmy.dbzer0.com ⁨5⁩ ⁨days⁩ ago
    You would probably need to use something like Visual Novel Character Creation Suite to generate a character and a dataset for LoRA training, and then train a LoRA for each character.
    
    If you’re trying to get started with generating locally, here’s a tutorial for Comfy UI.
    
    source
    -> View More Comments
tal@lemmy.today ⁨5⁩ ⁨days⁩ ago
My limited experience is that stable characters across a number of images are a weakness today, and I wouldn’t be confident that genAI is a great way to go about it. If you want to try it, here’s what I’d go with:

If you can get images with consistent outlines via some other route, you can try using ControlNet to do the rest of the image.

If you just need slight variations on a particular image, you can use inpainting to regenerate the relevant portions (e.g. an image with a series of different expressions).

If you want to work from a prompt, try picking a real-life person or character as a starting point, that may help, as models have been trained on them. Best is if you can get them at once point in time (e.g. “actor in popularmovie”). If you have a character description that you’re slapping into each prompt, only describe elements that are actually visible in a given image.

I’ve found that a consistent theme is something that is much more achievable, in that you can add “by <artist name>” to your prompt terms for any artist that the model has been trained on a number of images from. If you’re using a model that supports prompt term weighting (e.g. Stable Diffusion), you can increase the weight here to increase the strength of the effect. Flux doesn’t support prompt term weighting (though it’s really aimed at photographic images anyway). It’s possible to blend multiple artists or genres as prompt terms.
source
- nicgentile@lemmy.world ⁨5⁩ ⁨days⁩ ago
  Eyevballing here. I’m a learner and have hardly used this.
  
  Would a checkpoint model (right term?) achieve consistency, built from a specific set of pictures?
  
  source
  - tal@lemmy.today ⁨5⁩ ⁨days⁩ ago
    Yes, if you have or can create a LoRA trained on images of the character you’re presenting.
    
    source
  - Even_Adder@lemmy.dbzer0.com ⁨5⁩ ⁨days⁩ ago
    You might be thinking of a LoRA. LoRA are adapters you use with larger models to help it generate whatever concept you want.
    
    source
    -> View More Comments