SWT's AI prompting and training endeavours. Part 1 by SWTrium
Members remain the original copyright holder in all their materials here at Renderosity. Use of any of their material inconsistent with the terms and conditions set forth is prohibited and is considered an infringement of the copyrights of the respective holders unless specially stated otherwise.
Description
Since this post can get very technical and complicated very easily, I will not go into the minute details to prevent slipping down that rabbit hole. But here are some sample previews of my attempts at training my own datachunks (called LoRA) to use inside Stable Diffusion AI models to create AI generative content.
In this first serious attempt at training the AI models using my own 3D creations and renders, I am using my Drow Mercenary kit (the armor set which can be found on : https://www.renderosity.com/marketplace/products/145253/dforce-drow-mercenary-for-genesis-8-females) mainly because this is one of the most intricate kits which I have in my catalogue and my reasoning was that if I can train the machine with a difficult concept then the easier concepts will be so much more effortless.
For training related tasks I first rendered out a few images and used the older promos that I had already created for these kits. And then got to work creating the LoRA which I am calling "Dinae", based on the name of my OC who I used for the Drow Mercenary's main promo images. I spent a few tries with different settings, out of three runs I was happy with the second version and these previews have been created using that particular version (Dinae_V2).
The AI images seen over here, have been created with the sole keyword "woman" using my LoRA trained on the Drow Mercenary armor+character details, along with the usual set of negative prompts (like worst quality, normal quality, ugly, deformed, etc.). I used a few different AI models to try the LoRA's efficacy with various styles and weights, but I was happy to see that the LoRA trained on Stable Diffusion 1.5 model worked pretty nicely on both realistic, 2.5D semi-realistic as well as cartoon/anime AI models with minimal effort.
Samplers used here are mainly UniPC with some Euler A, IIRC. CodeFormer face correction used on the 3D images. Other settings vary.
I'm still refining this LoRA before moving onto creating LoRAs for some of my other outfits, characters and maybe some new concepts which are not currently easily available. When I'm happy with the quality, I intend to share some LoRAs (or whatever will be the current working format at that point in time) with others since I fear I've seen my work get used by others to create their own LoRAs so it doesn't make sense that I shouldn't be using them myself to create freebies for others as the original creator of the 3D assets and rendered content. I think it's high time to stop take a passive approach to these things, since the market disruptor has probably arrived, so here's my first attempt and doing stuff.
I've still got a long way to go, since I do not have any background in Machine Learning or AI training this is all a hand's on approach for me with a lot of testing and reading up on other's experiences in the field. But in the same way how I started 3D development, I'll give this a shot since I find it fun, and will let some trial and error type of experience guide me onward.
End of a lengthy post. The subsequent ones will not be as rant-y as this but should contain more images and stuff.
Credits:
This Drow Mercenary kit was commissioned by Cederien (https://www.deviantart.com/cederien) for his own OC and to create the 3D assets I used concept art and 2D illustrations supplied by him which he commissioned from SimmotoSyoshiko (https://www.deviantart.com/simmotosyoshiko) to tailor it to his (or his OC's actually) needs. The 3D assets and renders have been created by yours truly.
Comments (1)
randalthor2
Training AIs is a still some ways ahead for me, but for whatever it's worth I think the results above are pretty awesome. The armor looks quite identical and if the char and armor can be that easily summoned I think that's really great. Reduces the guesswork in the prompting.
All in all, a solid effort and wish you further success in the project!
SWTrium
I was pretty happy with the outcome too, especially since I didn't have too many renders readily available to use as a training set. So with just a small amount the machine was able to learn some things, but it's not as flexible as I would like it to be yet (can't successfully mix too many other outfit LoRAs or color details on the current bake). But I'm currently lacking the hardware to push the training towards higher resolution bases for something as intricate as this Drow Merc set. Once that issue with the VRAM gets resolved in the coming days, the experiments will continue.
Sadly I have to take a break from training to focus on creating my backlogged 3D assets, but your wishes are graciously accepted and hopefully the next batch bears better fruits! :)