Revert "add styles for image zoom"

This reverts commit 326236ed31.

.gitignore

@@ -4,6 +4,7 @@ __pycache__
 /venv
 /tmp
 /model.ckpt
+/models/*.ckpt
 /GFPGANv1.3.pth
 /ui-config.json
 /outputs
@@ -14,4 +15,6 @@ __pycache__
 /styles.csv
 /styles.csv.bak
 /webui-user.bat
+/webui-user.sh
 /interrogate
+/user.css

README.md

@@ -3,10 +3,8 @@ A browser interface based on Gradio library for Stable Diffusion.
 
 ![](screenshot.png)
 
-## Feature showcase
-
-[Detailed feature showcase with images, art by Greg Rutkowski](https://github.com/AUTOMATIC1111/stable-diffusion-webui-feature-showcase)
-
+## Features
+[Detailed feature showcase with images](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features):
 - Original txt2img and img2img modes
 - One click install and run script (but you still must install python and git)
 - Outpainting
@@ -18,10 +16,10 @@ A browser interface based on Gradio library for Stable Diffusion.
 - X/Y plot
 - Textual Inversion
 - Extras tab with:
-  - GFPGAN, neural network that fixes faces
-  - CodeFormer, face restoration tool as an alternative to GFPGAN
-  - RealESRGAN, neural network upscaler
-  - ESRGAN, neural network with a lot of third party models
+    - GFPGAN, neural network that fixes faces
+    - CodeFormer, face restoration tool as an alternative to GFPGAN
+    - RealESRGAN, neural network upscaler
+    - ESRGAN, neural network with a lot of third party models
 - Resizing aspect ratio options
 - Sampling method selection
 - Interrupt processing at any time
@@ -42,283 +40,38 @@ A browser interface based on Gradio library for Stable Diffusion.
 - Variations
 - Seed resizing
 - CLIP interrogator
+- Prompt Editing
 
-## Installing and running
+## Installation and Running
+Make sure the required [dependencies](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Dependencies) are met and follow the instructions available for both [NVidia](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs) (recommended) and [AMD](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs) GPUs.
 
-You need [python](https://www.python.org/downloads/windows/) and [git](https://git-scm.com/download/win)
-installed to run this, and an NVidia video card.
+Alternatively, use [Google Colab](https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxueGij8phCyluOh).
 
-You need `model.ckpt`, Stable Diffusion model checkpoint, a big file containing the neural network weights. You
-can obtain it from the following places:
- - [official download](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original)
- - [file storage](https://drive.yerf.org/wl/?id=EBfTrmcCCUAGaQBXVIj5lJmEhjoP1tgl)
- - magnet:?xt=urn:btih:3a4a612d75ed088ea542acac52f9f45987488d1c&dn=sd-v1-4.ckpt&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337
-
-You can optionally use GFPGAN to improve faces, to do so you'll need to download the model from [here](https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth) and place it in the same directory as `webui.bat`.
-
-To use ESRGAN models, put them into ESRGAN directory in the same location as webui.py. A file will be loaded
-as a model if it has .pth extension, and it will show up with its name in the UI. Grab models from the [Model Database](https://upscale.wiki/wiki/Model_Database).
-
-> Note: RealESRGAN models are not ESRGAN models, they are not compatible. Do not download RealESRGAN models. Do not place
-RealESRGAN into the directory with ESRGAN models. Thank you.
-
-### Automatic installation/launch
-
-- install [Python 3.10.6](https://www.python.org/downloads/windows/) and check "Add Python to PATH" during installation. You must install this exact version.
-- install [git](https://git-scm.com/download/win)
-- place `model.ckpt` into webui directory, next to `webui.bat`.
-- _*(optional)*_ place `GFPGANv1.3.pth` into webui directory, next to `webui.bat`.
-- run `webui-user.bat` from Windows Explorer. Run it as a normal user, ***not*** as administrator.
-
-#### Troubleshooting
-
-- if your version of Python is not in PATH (or if another version is), edit `webui-user.bat`, and modify the
-line `set PYTHON=python` to say the full path to your python executable, for example: `set PYTHON=B:\soft\Python310\python.exe`.
-You can do this for python, but not for git.
-- if you get out of memory errors and your video-card has a low amount of VRAM (4GB), use custom parameter `set COMMANDLINE_ARGS` (see section below)
-to enable appropriate optimization according to low VRAM guide below (for example, `set COMMANDLINE_ARGS=--medvram --opt-split-attention`).
-- to prevent the creation of virtual environment and use your system python, use custom parameter replacing `set VENV_DIR=-` (see below).
-- webui.bat installs requirements from files `requirements_versions.txt`, which lists versions for modules specifically compatible with
-Python 3.10.6. If you choose to install for a different version of python,  using custom parameter `set REQS_FILE=requirements.txt`
-may help (but I still recommend you to just use the recommended version of python).
-- if you feel you broke something and want to reinstall from scratch, delete directories: `venv`, `repositories`.
-- if you get a green or black screen instead of generated pictures, you have a card that doesn't support half precision
-floating point numbers (Known issue with 16xx cards). You must use `--precision full --no-half` in addition to command line
-arguments (set them using `set COMMANDLINE_ARGS`, see below), and the model will take much more space in VRAM (you will likely
-have to also use at least `--medvram`).
-- the installer creates a python virtual environment, so none of the installed modules will affect your system installation of python if
-you had one prior to installing this.
-- About _"You must install this exact version"_ from the instructions above: you can use any version of python you like,
-and it will likely work, but if you want to seek help about things not working, I will not offer help unless you use this
-exact version for my sanity.
-
-#### How to run with custom parameters
-
-It's possible to edit `set COMMANDLINE_ARGS=` line in `webui.bat` to run the program with different command line arguments, but that may lead
-to inconveniences when the file is updated in the repository.
-
-The recommended way is to use another .bat file named anything you like, set the parameters you want in it, and run webui.bat from it.
-A `webui-user.bat` file included into the repository does exactly this.
-
-Here is an example that runs the program with `--opt-split-attention` argument:
-
-```commandline
-@echo off
-
-set COMMANDLINE_ARGS=--opt-split-attention
-
-call webui.bat
-```
-
-Another example, this file will run the program with a custom python path, a different model named `a.ckpt` and without a virtual environment:
-
-```commandline
-@echo off
-
-set PYTHON=b:/soft/Python310/Python.exe
-set VENV_DIR=-
-set COMMANDLINE_ARGS=--ckpt a.ckpt
-
-call webui.bat
-```
-
-### How to create large images?
-Use `--opt-split-attention` parameter. It slows down sampling a tiny bit, but allows you to make gigantic images.
-
-### What options to use for low VRAM video-cards?
-You can, through command line arguments, enable the various optimizations which sacrifice some/a lot of speed in favor of
-using less VRAM. Those arguments are added to the `COMMANDLINE_ARGS` parameter, see section above.
-
-Here's a list of optimization arguments:
-- If you have 4GB VRAM and want to make 512x512 (or maybe up to 640x640) images, use `--medvram`.
-- If you have 4GB VRAM and want to make 512x512 images, but you get an out of memory error with `--medvram`, use `--medvram --opt-split-attention` instead.
-- If you have 4GB VRAM and want to make 512x512 images, and you still get an out of memory error, use `--lowvram --always-batch-cond-uncond --opt-split-attention` instead.
-- If you have 4GB VRAM and want to make images larger than you can with `--medvram`, use  `--lowvram --opt-split-attention`.
-- If you have more VRAM and want to make larger images than you can usually make (for example 1024x1024 instead of 512x512), use `--medvram --opt-split-attention`. You can use `--lowvram`
-also but the effect will likely be barely noticeable.
-- Otherwise, do not use any of those.
-
-### Running online
-
-Use the `--share` option to run online. You will get a xxx.app.gradio link. This is the intended way to use the
-program in collabs. You may set up authentication for said gradio shared instance with the flag `--gradio-auth username:password`, optionally providing multiple sets of usernames and passwords separated by commas.
-
-Use `--listen` to make the server listen to network connections. This will allow computers on the local network
-to access the UI, and if you configure port forwarding, also computers on the internet.
-
-Use `--port xxxx` to make the server listen on a specific port, xxxx being the wanted port. Remember that
-all ports below 1024 need root/admin rights, for this reason it is advised to use a port above 1024.
-Defaults to port 7860 if available.
-
-### Google collab
-
-If you don't want or can't run locally, here is a Google colab that allows you to run the webui:
-
-https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxueGij8phCyluOh
-
-### Textual Inversion
-To make use of pretrained embeddings, create an `embeddings` directory (in the same place as `webui.py`)
-and put your embeddings into it. They must be either .pt or .bin files, each with only one trained embedding,
-and the filename (without .pt/.bin) will be the term you'll use in the prompt to get that embedding.
-
-As an example, I trained one for about 5000 steps: https://files.catbox.moe/e2ui6r.pt; it does not produce
-very good results, but it does work. To try it out download the file, rename it to `Usada Pekora.pt`, put it into the `embeddings` dir
-and use `Usada Pekora` in the prompt.
-
-You may also try some from the growing library of embeddings at https://huggingface.co/sd-concepts-library, downloading one of the `learned_embeds.bin` files, renaming it to the term you want to use for it in the prompt (be sure to keep the .bin extension) and putting it in your `embeddings` directory.
-
-### How to change UI defaults?
-
-After running once, a `ui-config.json` file appears in webui directory:
-
-```json
-{
-    "txt2img/Sampling Steps/value": 20,
-    "txt2img/Sampling Steps/minimum": 1,
-    "txt2img/Sampling Steps/maximum": 150,
-    "txt2img/Sampling Steps/step": 1,
-    "txt2img/Batch count/value": 1,
-    "txt2img/Batch count/minimum": 1,
-    "txt2img/Batch count/maximum": 32,
-    "txt2img/Batch count/step": 1,
-    "txt2img/Batch size/value": 1,
-    "txt2img/Batch size/minimum": 1,
-```
-
-Edit values to your liking and the next time you launch the program they will be applied.
-
-### Almost automatic installation and launch
-
-Install python and git, place `model.ckpt` and `GFPGANv1.3.pth` into webui directory, run:
-
-```
-python launch.py
-```
-
-This installs packages via pip. If you need to use a virtual environment, you must set it up yourself. I will not
-provide support for using the web ui this way unless you are using the recommended version of python below.
-
-If you'd like to use command line parameters, use them right there:
-
-```
-python launch.py --opt-split-attention --ckpt ../secret/anime9999.ckpt
-```
-
-### Manual installation
-Alternatively, if you don't want to run the installer, here are instructions for installing
-everything by hand. This can run on both Windows and Linux (if you're on linux, use `ls`
-instead of `dir`). 
+### Automatic Installation on Windows
+1. Install [Python 3.10.6](https://www.python.org/downloads/windows/), checking "Add Python to PATH"
+2. Install [git](https://git-scm.com/download/win).
+3. Download the stable-diffusion-webui repository, for example by running `git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git`.
+4. Place `model.ckpt` in the `models` directory (see [dependencies](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Dependencies) for where to get it).
+5. _*(Optional)*_ Place `GFPGANv1.3.pth` in the base directory, alongside `webui.py` (see [dependencies](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Dependencies) for where to get it).
+6. Run `webui-user.bat` from Windows Explorer as normal, non-administrator, user.
 
+### Automatic Installation on Linux
+1. Install the dependencies:
 ```bash
-# install torch with CUDA support. See https://pytorch.org/get-started/locally/ for more instructions if this fails.
-pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
-
-# check if torch supports GPU; this must output "True". You need CUDA 11. installed for this. You might be able to use
-# a different version, but this is what I tested.
-python -c "import torch; print(torch.cuda.is_available())"
-
-# clone web ui and go into its directory
-git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
-cd stable-diffusion-webui
-
-# clone repositories for Stable Diffusion and (optionally) CodeFormer
-mkdir repositories
-git clone https://github.com/CompVis/stable-diffusion.git repositories/stable-diffusion
-git clone https://github.com/CompVis/taming-transformers.git repositories/taming-transformers
-git clone https://github.com/sczhou/CodeFormer.git repositories/CodeFormer
-git clone https://github.com/salesforce/BLIP.git repositories/BLIP
-
-# install requirements of Stable Diffusion
-pip install transformers==4.19.2 diffusers invisible-watermark --prefer-binary
-
-# install k-diffusion
-pip install git+https://github.com/crowsonkb/k-diffusion.git --prefer-binary
-
-# (optional) install GFPGAN (face restoration)
-pip install git+https://github.com/TencentARC/GFPGAN.git --prefer-binary
-
-# (optional) install requirements for CodeFormer (face restoration)
-pip install -r repositories/CodeFormer/requirements.txt --prefer-binary
-
-# install requirements of web ui
-pip install -r requirements.txt  --prefer-binary
-
-# update numpy to latest version
-pip install -U numpy  --prefer-binary
-
-# (outside of command line) put stable diffusion model into web ui directory
-# the command below must output something like: 1 File(s) 4,265,380,512 bytes
-dir model.ckpt
-
-# (outside of command line) put the GFPGAN model into web ui directory
-# the command below must output something like: 1 File(s) 348,632,874 bytes
-dir GFPGANv1.3.pth
+# Debian-based:
+sudo apt install wget git python3 python3-venv
+# Red Hat-based:
+sudo dnf install wget git python3
+# Arch-based:
+sudo pacman -S wget git python3
 ```
-
-> Note: the directory structure for manual instruction has been changed on 2022-09-09 to match automatic installation: previously
-> webui was in a subdirectory of stable diffusion, now it's the reverse. If you followed manual installation before the
-> change, you can still use the program with your existing directory structure.
-
-After that the installation is finished.
-
-Run the command to start web ui:
-
-```
-python webui.py
-```
-
-If you have a 4GB video card, run the command with either `--lowvram` or `--medvram` argument:
-
-```
-python webui.py --medvram
-```
-
-After a while, you will get a message like this:
-
-```
-Running on local URL:  http://127.0.0.1:7860/
-```
-
-Open the URL in a browser, and you are good to go.
-
-
-### Windows 11 WSL2 instructions
-Alternatively, here are instructions for installing under Windows 11 WSL2 Linux distro, everything by hand:
-
+2. To install in `/home/$(whoami)/stable-diffusion-webui/`, run:
 ```bash
-# install conda (if not already done)
-wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
-chmod +x Anaconda3-2022.05-Linux-x86_64.sh 
-./Anaconda3-2022.05-Linux-x86_64.sh
-
-# Clone webui repo
-git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
-cd stable-diffusion-webui
-
-# Create and activate conda env
-conda env create -f environment-wsl2.yaml
-conda activate automatic
-
-# (optional) install requirements for GFPGAN (upscaling)
-wget https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth
+bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)
 ```
 
-After that follow the instructions in the `Manual instructions` section starting at step `:: clone repositories for Stable Diffusion and (optionally) CodeFormer`.
-
-### Custom scripts from users
-
-[A list of custom scripts](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Custom-scripts-from-users), along with installation instructions.
-
-### img2img alternative test
-- see [this post](https://www.reddit.com/r/StableDiffusion/comments/xboy90/a_better_way_of_doing_img2img_by_finding_the/) on ebaumsworld.com for context.
-- find it in scripts section
-- put description of input image into the Original prompt field
-- use Euler only
-- recommended: 50 steps, low cfg scale between 1 and 2
-- denoising and seed don't matter
-- decode cfg scale between 0 and 1
-- decode steps 50
-- original blue haired woman close nearly reproduces with cfg scale=1.8
+## Documentation
+The documentation was moved from this README over to the project's [wiki](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki).
 
 ## Credits
 - Stable Diffusion - https://github.com/CompVis/stable-diffusion, https://github.com/CompVis/taming-transformers
@@ -326,8 +79,9 @@ After that follow the instructions in the `Manual instructions` section starting
 - GFPGAN - https://github.com/TencentARC/GFPGAN.git
 - ESRGAN - https://github.com/xinntao/ESRGAN
 - Ideas for optimizations - https://github.com/basujindal/stable-diffusion
-- Cross Attention layer optimization - https://github.com/Doggettx/stable-diffusion
+- Doggettx - Cross Attention layer optimization - https://github.com/Doggettx/stable-diffusion, original idea for prompt editing.
 - Idea for SD upscale - https://github.com/jquesnelle/txt2imghd
+- Noise generation for outpainting mk2 - https://github.com/parlance-zz/g-diffuser-bot
 - CLIP interrogator idea and borrowing some code - https://github.com/pharmapsychotic/clip-interrogator
 - Initial Gradio script - posted on 4chan by an Anonymous user. Thank you Anonymous user.
 - (You)

javascript/dragdrop.js

@@ -0,0 +1,59 @@
+// allows drag-dropping files into gradio image elements, and also pasting images from clipboard
+
+function isValidImageList( files ) {
+    return files && files?.length === 1 && ['image/png', 'image/gif', 'image/jpeg'].includes(files[0].type);
+}
+
+function dropReplaceImage( imgWrap, files ) {
+    if ( ! isValidImageList( files ) ) {
+        return;
+    }
+
+    imgWrap.querySelector('.modify-upload button + button, .touch-none + div button + button')?.click();
+    window.requestAnimationFrame( () => {
+        const fileInput = imgWrap.querySelector('input[type="file"]');
+        if ( fileInput ) {
+            fileInput.files = files;
+            fileInput.dispatchEvent(new Event('change'));   
+        }
+    });
+}
+
+window.document.addEventListener('dragover', e => {
+    const target = e.composedPath()[0];
+    const imgWrap = target.closest('[data-testid="image"]');
+    if ( !imgWrap ) {
+        return;
+    }
+    e.stopPropagation();
+    e.preventDefault();
+    e.dataTransfer.dropEffect = 'copy';
+});
+
+window.document.addEventListener('drop', e => {
+    const target = e.composedPath()[0];
+    const imgWrap = target.closest('[data-testid="image"]');
+    if ( !imgWrap ) {
+        return;
+    }
+    e.stopPropagation();
+    e.preventDefault();
+    const files = e.dataTransfer.files;
+    dropReplaceImage( imgWrap, files );
+});
+
+window.addEventListener('paste', e => {
+    const files = e.clipboardData.files;
+    if ( ! isValidImageList( files ) ) {
+        return;
+    }
+    [...gradioApp().querySelectorAll('input[type=file][accept="image/x-png,image/gif,image/jpeg"]')]
+        .filter(input => !input.matches('.\\!hidden input[type=file]'))
+        .forEach(input => {
+            input.files = files;
+            input.dispatchEvent(new Event('change'))
+        });
+    [...gradioApp().querySelectorAll('[data-testid="image"]')]
+        .filter(imgWrap => !imgWrap.closest('.\\!hidden'))
+        .forEach(imgWrap => dropReplaceImage( imgWrap, files ));
+});

javascript/hints.js

@@ -0,0 +1,111 @@
+// mouseover tooltips for various UI elements
+
+titles = {
+    "Sampling steps": "How many times to improve the generated image iteratively; higher values take longer; very low values can produce bad results",
+    "Sampling method": "Which algorithm to use to produce the image",
+	"GFPGAN": "Restore low quality faces using GFPGAN neural network",
+	"Euler a": "Euler Ancestral - very creative, each can get a completely different picture depending on step count, setting steps to higher than 30-40 does not help",
+	"DDIM": "Denoising Diffusion Implicit Models - best at inpainting",
+
+	"Batch count": "How many batches of images to create",
+	"Batch size": "How many image to create in a single batch",
+    "CFG Scale": "Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results",
+    "Seed": "A value that determines the output of random number generator - if you create an image with same parameters and seed as another image, you'll get the same result",
+    "\u{1f3b2}\ufe0f": "Set seed to -1, which will cause a new random number to be used every time",
+    "\u267b\ufe0f": "Reuse seed from last generation, mostly useful if it was randomed",
+
+    "Inpaint a part of image": "Draw a mask over an image, and the script will regenerate the masked area with content according to prompt",
+    "SD upscale": "Upscale image normally, split result into tiles, improve each tile using img2img, merge whole image back",
+
+    "Just resize": "Resize image to target resolution. Unless height and width match, you will get incorrect aspect ratio.",
+    "Crop and resize": "Resize the image so that entirety of target resolution is filled with the image. Crop parts that stick out.",
+    "Resize and fill": "Resize the image so that entirety of image is inside target resolution. Fill empty space with image's colors.",
+
+    "Mask blur": "How much to blur the mask before processing, in pixels.",
+    "Masked content": "What to put inside the masked area before processing it with Stable Diffusion.",
+    "fill": "fill it with colors of the image",
+    "original": "keep whatever was there originally",
+    "latent noise": "fill it with latent space noise",
+    "latent nothing": "fill it with latent space zeroes",
+    "Inpaint at full resolution": "Upscale masked region to target resolution, do inpainting, downscale back and paste into original image",
+
+    "Denoising strength": "Determines how little respect the algorithm should have for image's content. At 0, nothing will change, and at 1 you'll get an unrelated image. With values below 1.0, processing will take less steps than the Sampling Steps slider specifies.",
+    "Denoising strength change factor": "In loopback mode, on each loop the denoising strength is multiplied by this value. <1 means decreasing variety so your sequence will converge on a fixed picture. >1 means increasing variety so your sequence will become more and more chaotic.",
+
+    "Interrupt": "Stop processing images and return any results accumulated so far.",
+    "Save": "Write image to a directory (default - log/images) and generation parameters into csv file.",
+
+    "X values": "Separate values for X axis using commas.",
+    "Y values": "Separate values for Y axis using commas.",
+
+    "None": "Do not do anything special",
+    "Prompt matrix": "Separate prompts into parts using vertical pipe character (|) and the script will create a picture for every combination of them (except for the first part, which will be present in all combinations)",
+    "X/Y plot": "Create a grid where images will have different parameters. Use inputs below to specify which parameters will be shared by columns and rows",
+    "Custom code": "Run Python code. Advanced user only. Must run program with --allow-code for this to work",
+
+    "Prompt S/R": "Separate a list of words with commas, and the first word will be used as a keyword: script will search for this word in the prompt, and replace it with others",
+
+    "Tiling": "Produce an image that can be tiled.",
+    "Tile overlap": "For SD upscale, how much overlap in pixels should there be between tiles. Tiles overlap so that when they are merged back into one picture, there is no clearly visible seam.",
+
+    "Roll": "Add a random artist to the prompt.",
+
+    "Variation seed": "Seed of a different picture to be mixed into the generation.",
+    "Variation strength": "How strong of a variation to produce. At 0, there will be no effect. At 1, you will get the complete picture with variation seed (except for ancestral samplers, where you will just get something).",
+    "Resize seed from height": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution",
+    "Resize seed from width": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution",
+
+    "Interrogate": "Reconstruct prompt from existing image and put it into the prompt field.",
+
+    "Images filename pattern": "Use following tags to define how filenames for images are chosen: [steps], [cfg], [prompt], [prompt_spaces], [width], [height], [sampler], [seed], [model_hash], [prompt_words], [date]; leave empty for default.",
+    "Directory name pattern": "Use following tags to define how subdirectories for images and grids are chosen: [steps], [cfg], [prompt], [prompt_spaces], [width], [height], [sampler], [seed], [model_hash], [prompt_words], [date]; leave empty for default.",
+
+    "Loopback": "Process an image, use it as an input, repeat.",
+    "Loops": "How many times to repeat processing an image and using it as input for the next iteration",
+
+
+    "Style 1": "Style to apply; styles have components for both positive and negative prompts and apply to both",
+    "Style 2": "Style to apply; styles have components for both positive and negative prompts and apply to both",
+    "Apply style": "Insert selected styles into prompt fields",
+    "Create style": "Save current prompts as a style. If you add the token {prompt} to the text, the style use that as placeholder for your prompt when you use the style in the future.",
+
+    "Checkpoint name": "Loads weights from checkpoint before making images. You can either use hash or a part of filename (as seen in settings) for checkpoint name. Recommended to use with Y axis for less switching.",
+
+    "vram": "Torch active: Peak amount of VRAM used by Torch during generation, excluding cached data.\nTorch reserved: Peak amount of VRAM allocated by Torch, including all active and cached data.\nSys VRAM: Peak amount of VRAM allocation across all applications / total GPU VRAM (peak utilization%).",
+
+    "Highres. fix": "Use a two step process to partially create an image at smaller resolution, upscale, and then improve details in it without changing composition",
+    "Scale latent": "Uscale the image in latent space. Alternative is to produce the full image from latent representation, upscale that, and then move it back to latent space.",
+
+}
+
+
+onUiUpdate(function(){
+	gradioApp().querySelectorAll('span, button, select, p').forEach(function(span){
+		tooltip = titles[span.textContent];
+
+		if(!tooltip){
+		    tooltip = titles[span.value];
+		}
+
+		if(!tooltip){
+			for (const c of span.classList) {
+				if (c in titles) {
+					tooltip = titles[c];
+					break;
+				}
+			}
+		}
+
+		if(tooltip){
+			span.title = tooltip;
+		}
+	})
+
+	gradioApp().querySelectorAll('select').forEach(function(select){
+	    if (select.onchange != null) return;
+
+	    select.onchange = function(){
+            select.title = titles[select.value] || "";
+	    }
+	})
+})

javascript/imageviewer.js

@@ -0,0 +1,139 @@
+// A full size 'lightbox' preview modal shown when left clicking on gallery previews
+
+function closeModal() {
+  gradioApp().getElementById("lightboxModal").style.display = "none";
+}
+
+function showModal(event) {
+  var source = event.target || event.srcElement;
+  gradioApp().getElementById("modalImage").src = source.src
+  var lb = gradioApp().getElementById("lightboxModal")
+  lb.style.display = "block";
+  lb.focus()
+  event.stopPropagation()
+}
+
+function negmod(n, m) {
+  return ((n % m) + m) % m;
+}
+
+function modalImageSwitch(offset){
+  var galleryButtons = gradioApp().querySelectorAll(".gallery-item.transition-all")
+
+  if(galleryButtons.length>1){
+      var currentButton  = gradioApp().querySelector(".gallery-item.transition-all.\\!ring-2")
+
+      var result = -1
+      galleryButtons.forEach(function(v, i){ if(v==currentButton) { result = i } })
+
+      if(result != -1){
+        nextButton = galleryButtons[negmod((result+offset),galleryButtons.length)]
+        nextButton.click()
+        gradioApp().getElementById("modalImage").src = nextButton.children[0].src
+        setTimeout( function(){gradioApp().getElementById("lightboxModal").focus()},10)
+      }
+  }
+}
+
+function modalNextImage(event){
+  modalImageSwitch(1)
+  event.stopPropagation()
+}
+
+function modalPrevImage(event){
+  modalImageSwitch(-1)  
+  event.stopPropagation()
+}
+
+function modalKeyHandler(event){
+    switch (event.key) {
+        case "ArrowLeft":
+            modalPrevImage(event)
+            break;
+        case "ArrowRight":
+            modalNextImage(event)
+            break;
+        case "Escape":
+            closeModal();
+            break;
+    }
+}
+
+function showGalleryImage(){
+    setTimeout(function() {
+        fullImg_preview = gradioApp().querySelectorAll('img.w-full.object-contain')
+        
+        if(fullImg_preview != null){
+            fullImg_preview.forEach(function function_name(e) {
+                if(e && e.parentElement.tagName == 'DIV'){
+
+                    e.style.cursor='pointer'
+
+                    e.addEventListener('click', function (evt) {
+                        if(!opts.js_modal_lightbox) return;
+                        showModal(evt)
+                    },true);
+                }
+            });
+        }
+
+    }, 100);
+}
+
+function galleryImageHandler(e){
+    if(e && e.parentElement.tagName == 'BUTTON'){
+        e.onclick = showGalleryImage;
+    }
+}
+
+onUiUpdate(function(){
+	fullImg_preview = gradioApp().querySelectorAll('img.w-full')
+	    if(fullImg_preview != null){
+		fullImg_preview.forEach(galleryImageHandler);
+	}
+})
+
+document.addEventListener("DOMContentLoaded", function() {
+    const modalFragment = document.createDocumentFragment();
+    const modal = document.createElement('div')
+    modal.onclick = closeModal;
+    
+    const modalClose = document.createElement('span')
+    modalClose.className = 'modalClose cursor';
+    modalClose.innerHTML = '×'
+    modalClose.onclick = closeModal;
+    modal.id = "lightboxModal";
+    modal.tabIndex=0
+    modal.addEventListener('keydown', modalKeyHandler, true)
+    modal.appendChild(modalClose)
+
+    const modalImage = document.createElement('img')
+    modalImage.id = 'modalImage';
+    modalImage.onclick = closeModal;
+    modalImage.tabIndex=0
+    modalImage.addEventListener('keydown', modalKeyHandler, true)
+    modal.appendChild(modalImage)
+
+    const modalPrev = document.createElement('a')
+    modalPrev.className = 'modalPrev';
+    modalPrev.innerHTML = '❮'
+    modalPrev.tabIndex=0
+    modalPrev.addEventListener('click',modalPrevImage,true);
+    modalPrev.addEventListener('keydown', modalKeyHandler, true)
+    modal.appendChild(modalPrev)
+
+    const modalNext = document.createElement('a')
+    modalNext.className = 'modalNext';
+    modalNext.innerHTML = '❯'
+    modalNext.tabIndex=0
+    modalNext.addEventListener('click',modalNextImage,true);
+    modalNext.addEventListener('keydown', modalKeyHandler, true)
+
+    modal.appendChild(modalNext)
+
+
+    gradioApp().getRootNode().appendChild(modal)
+    
+    document.body.appendChild(modalFragment);
+	
+});

javascript/progressbar.js

@@ -0,0 +1,46 @@
+// code related to showing and updating progressbar shown as the image is being made
+
+global_progressbar = null
+
+onUiUpdate(function(){
+	progressbar = gradioApp().getElementById('progressbar')
+	if(progressbar!= null && progressbar != global_progressbar){
+	    global_progressbar = progressbar
+
+        var mutationObserver = new MutationObserver(function(m){
+            txt2img_preview = gradioApp().getElementById('txt2img_preview')
+            txt2img_gallery = gradioApp().getElementById('txt2img_gallery')
+
+            img2img_preview = gradioApp().getElementById('img2img_preview')
+            img2img_gallery = gradioApp().getElementById('img2img_gallery')
+
+            if(txt2img_preview != null && txt2img_gallery != null){
+                txt2img_preview.style.width = txt2img_gallery.clientWidth + "px"
+                txt2img_preview.style.height = txt2img_gallery.clientHeight + "px"
+            }
+
+            if(img2img_preview != null && img2img_gallery != null){
+                img2img_preview.style.width = img2img_gallery.clientWidth + "px"
+                img2img_preview.style.height = img2img_gallery.clientHeight + "px"
+            }
+
+            window.setTimeout(requestMoreProgress, 500)
+        });
+        mutationObserver.observe( progressbar, { childList:true, subtree:true })
+	}
+})
+
+function requestMoreProgress(){
+    btn = gradioApp().getElementById("check_progress");
+    if(btn==null) return;
+
+    btn.click();
+}
+
+function requestProgress(){
+    btn = gradioApp().getElementById("check_progress_initial");
+    if(btn==null) return;
+
+    btn.click();
+}
+

javascript/ui.js

@@ -0,0 +1,100 @@
+// various functions for interation with ui.py not large enough to warrant putting them in separate files
+
+function selected_gallery_index(){
+    var gr = gradioApp()
+    var buttons = gradioApp().querySelectorAll(".gallery-item")
+    var button = gr.querySelector(".gallery-item.\\!ring-2")
+
+    var result = -1
+    buttons.forEach(function(v, i){ if(v==button) { result = i } })
+
+    return result
+}
+
+function extract_image_from_gallery(gallery){
+    if(gallery.length == 1){
+        return gallery[0]
+    }
+
+    index = selected_gallery_index()
+
+    if (index < 0 || index >= gallery.length){
+        return [null]
+    }
+
+    return gallery[index];
+}
+
+function extract_image_from_gallery_img2img(gallery){
+    gradioApp().querySelectorAll('button')[1].click();
+    return extract_image_from_gallery(gallery);
+}
+
+function extract_image_from_gallery_extras(gallery){
+    gradioApp().querySelectorAll('button')[2].click();
+    return extract_image_from_gallery(gallery);
+}
+
+function submit(){
+    // this calls a function from progressbar.js
+    requestProgress()
+
+    res = []
+    for(var i=0;i<arguments.length;i++){
+        res.push(arguments[i])
+    }
+
+    // As it is currently, txt2img and img2img send back the previous output args (txt2img_gallery, generation_info, html_info) whenever you generate a new image.
+    // This can lead to uploading a huge gallery of previously generated images, which leads to an unnecessary delay between submitting and beginning to generate.
+    // I don't know why gradio is seding outputs along with inputs, but we can prevent sending the image gallery here, which seems to be an issue for some.
+    // If gradio at some point stops sending outputs, this may break something
+    if(Array.isArray(res[res.length - 3])){
+        res[res.length - 3] = null
+    }
+
+    return res
+}
+
+function ask_for_style_name(_, prompt_text, negative_prompt_text) {
+    name_ = prompt('Style name:')
+    return name_ === null ? [null, null, null]: [name_, prompt_text, negative_prompt_text]
+}
+
+opts = {}
+function apply_settings(jsdata){
+    console.log(jsdata)
+
+    opts = JSON.parse(jsdata)
+
+    return jsdata
+}
+
+onUiUpdate(function(){
+	if(Object.keys(opts).length != 0) return;
+
+	json_elem = gradioApp().getElementById('settings_json')
+	if(json_elem == null) return;
+
+    textarea = json_elem.querySelector('textarea')
+    jsdata = textarea.value
+    opts = JSON.parse(jsdata)
+
+
+    Object.defineProperty(textarea, 'value', {
+        set: function(newValue) {
+            var valueProp = Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, 'value');
+            var oldValue = valueProp.get.call(textarea);
+            valueProp.set.call(textarea, newValue);
+
+            if (oldValue != newValue) {
+                opts = JSON.parse(textarea.value)
+            }
+        },
+        get: function() {
+            var valueProp = Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, 'value');
+            return valueProp.get.call(textarea);
+        }
+    });
+
+    json_elem.parentElement.style.display="none"
+})

modules/codeformer_model.py

@@ -47,13 +47,11 @@ def setup_codeformer():
             def __init__(self):
                 self.net = None
                 self.face_helper = None
-                if shared.device.type == 'mps': # CodeFormer currently does not support mps backend
-                    shared.device_codeformer = torch.device('cpu')
 
             def create_models(self):
 
                 if self.net is not None and self.face_helper is not None:
-                    self.net.to(shared.device)
+                    self.net.to(devices.device_codeformer)
                     return self.net, self.face_helper
 
                 net = net_class(dim_embd=512, codebook_size=1024, n_head=8, n_layers=9, connect_list=['32', '64', '128', '256']).to(devices.device_codeformer)
@@ -66,7 +64,7 @@ def setup_codeformer():
 
                 self.net = net
                 self.face_helper = face_helper
-                self.net.to(shared.device)
+                self.net.to(devices.device_codeformer)
 
                 return net, face_helper
 

modules/extras.py

@@ -7,71 +7,94 @@ import modules.gfpgan_model
 from modules.ui import plaintext_to_html
 import modules.codeformer_model
 import piexif
+import piexif.helper
 
 
 cached_images = {}
 
 
-def run_extras(image, gfpgan_visibility, codeformer_visibility, codeformer_weight, upscaling_resize, extras_upscaler_1, extras_upscaler_2, extras_upscaler_2_visibility):
+def run_extras(image, image_folder, gfpgan_visibility, codeformer_visibility, codeformer_weight, upscaling_resize, extras_upscaler_1, extras_upscaler_2, extras_upscaler_2_visibility):
     devices.torch_gc()
 
-    existing_pnginfo = image.info or {}
+    imageArr = []
 
-    image = image.convert("RGB")
-    info = ""
+    if image_folder != None:
+        if image != None:
+            print("Batch detected and single image detected, please only use one of the two. Aborting.")
+            return None
+        #convert file to pillow image
+        for img in image_folder:
+            image = Image.fromarray(np.array(Image.open(img)))
+            imageArr.append(image)
+
+    elif image != None:
+        if image_folder != None:
+            print("Batch detected and single image detected, please only use one of the two. Aborting.")
+            return None
+        else:
+            imageArr.append(image)
 
     outpath = opts.outdir_samples or opts.outdir_extras_samples
 
-    if gfpgan_visibility > 0:
-        restored_img = modules.gfpgan_model.gfpgan_fix_faces(np.array(image, dtype=np.uint8))
-        res = Image.fromarray(restored_img)
+    outputs = []
+    for image in imageArr:
+        existing_pnginfo = image.info or {}
 
-        if gfpgan_visibility < 1.0:
-            res = Image.blend(image, res, gfpgan_visibility)
+        image = image.convert("RGB")
+        info = ""
 
-        info += f"GFPGAN visibility:{round(gfpgan_visibility, 2)}\n"
-        image = res
+        if gfpgan_visibility > 0:
+            restored_img = modules.gfpgan_model.gfpgan_fix_faces(np.array(image, dtype=np.uint8))
+            res = Image.fromarray(restored_img)
 
-    if codeformer_visibility > 0:
-        restored_img = modules.codeformer_model.codeformer.restore(np.array(image, dtype=np.uint8), w=codeformer_weight)
-        res = Image.fromarray(restored_img)
+            if gfpgan_visibility < 1.0:
+                res = Image.blend(image, res, gfpgan_visibility)
 
-        if codeformer_visibility < 1.0:
-            res = Image.blend(image, res, codeformer_visibility)
+            info += f"GFPGAN visibility:{round(gfpgan_visibility, 2)}\n"
+            image = res
 
-        info += f"CodeFormer w: {round(codeformer_weight, 2)}, CodeFormer visibility:{round(codeformer_visibility)}\n"
-        image = res
+        if codeformer_visibility > 0:
+            restored_img = modules.codeformer_model.codeformer.restore(np.array(image, dtype=np.uint8), w=codeformer_weight)
+            res = Image.fromarray(restored_img)
 
-    if upscaling_resize != 1.0:
-        def upscale(image, scaler_index, resize):
-            small = image.crop((image.width // 2, image.height // 2, image.width // 2 + 10, image.height // 2 + 10))
-            pixels = tuple(np.array(small).flatten().tolist())
-            key = (resize, scaler_index, image.width, image.height, gfpgan_visibility, codeformer_visibility, codeformer_weight) + pixels
+            if codeformer_visibility < 1.0:
+                res = Image.blend(image, res, codeformer_visibility)
 
-            c = cached_images.get(key)
-            if c is None:
-                upscaler = shared.sd_upscalers[scaler_index]
-                c = upscaler.upscale(image, image.width * resize, image.height * resize)
-                cached_images[key] = c
+            info += f"CodeFormer w: {round(codeformer_weight, 2)}, CodeFormer visibility:{round(codeformer_visibility, 2)}\n"
+            image = res
 
-            return c
+        if upscaling_resize != 1.0:
+            def upscale(image, scaler_index, resize):
+                small = image.crop((image.width // 2, image.height // 2, image.width // 2 + 10, image.height // 2 + 10))
+                pixels = tuple(np.array(small).flatten().tolist())
+                key = (resize, scaler_index, image.width, image.height, gfpgan_visibility, codeformer_visibility, codeformer_weight) + pixels
 
-        info += f"Upscale: {round(upscaling_resize, 3)}, model:{shared.sd_upscalers[extras_upscaler_1].name}\n"
-        res = upscale(image, extras_upscaler_1, upscaling_resize)
+                c = cached_images.get(key)
+                if c is None:
+                    upscaler = shared.sd_upscalers[scaler_index]
+                    c = upscaler.upscale(image, image.width * resize, image.height * resize)
+                    cached_images[key] = c
 
-        if extras_upscaler_2 != 0 and extras_upscaler_2_visibility > 0:
-            res2 = upscale(image, extras_upscaler_2, upscaling_resize)
-            info += f"Upscale: {round(upscaling_resize, 3)}, visibility: {round(extras_upscaler_2_visibility, 3)}, model:{shared.sd_upscalers[extras_upscaler_2].name}\n"
-            res = Image.blend(res, res2, extras_upscaler_2_visibility)
+                return c
 
-        image = res
+            info += f"Upscale: {round(upscaling_resize, 3)}, model:{shared.sd_upscalers[extras_upscaler_1].name}\n"
+            res = upscale(image, extras_upscaler_1, upscaling_resize)
 
-    while len(cached_images) > 2:
-        del cached_images[next(iter(cached_images.keys()))]
+            if extras_upscaler_2 != 0 and extras_upscaler_2_visibility > 0:
+                res2 = upscale(image, extras_upscaler_2, upscaling_resize)
+                info += f"Upscale: {round(upscaling_resize, 3)}, visibility: {round(extras_upscaler_2_visibility, 3)}, model:{shared.sd_upscalers[extras_upscaler_2].name}\n"
+                res = Image.blend(res, res2, extras_upscaler_2_visibility)
 
-    images.save_image(image, outpath, "", None, info=info, extension=opts.samples_format, short_filename=True, no_prompt=True, pnginfo_section_name="extras", existing_info=existing_pnginfo)
+            image = res
 
-    return image, plaintext_to_html(info), ''
+        while len(cached_images) > 2:
+            del cached_images[next(iter(cached_images.keys()))]
+
+        images.save_image(image, path=outpath, basename="", seed=None, prompt=None, extension=opts.samples_format, info=info, short_filename=True, no_prompt=True, grid=False, pnginfo_section_name="extras", existing_info=existing_pnginfo)
+
+        outputs.append(image)
+
+    return outputs, plaintext_to_html(info), ''
 
 
 def run_pnginfo(image):
@@ -80,11 +103,17 @@ def run_pnginfo(image):
     if "exif" in image.info:
         exif = piexif.load(image.info["exif"])
         exif_comment = (exif or {}).get("Exif", {}).get(piexif.ExifIFD.UserComment, b'')
-        exif_comment = exif_comment.decode("utf8", 'ignore')
+        try:
+            exif_comment = piexif.helper.UserComment.load(exif_comment)
+        except ValueError:
+            exif_comment = exif_comment.decode('utf8', errors="ignore")
+
+
         items['exif comment'] = exif_comment
 
-        for field in ['jfif', 'jfif_version', 'jfif_unit', 'jfif_density', 'dpi', 'exif']:
-            del items[field]
+        for field in ['jfif', 'jfif_version', 'jfif_unit', 'jfif_density', 'dpi', 'exif',
+                      'loop', 'background', 'timestamp', 'duration']:
+            items.pop(field, None)
 
 
     info = ''

modules/images.py

@@ -13,7 +13,7 @@ import string
 
 import modules.shared
 from modules import sd_samplers, shared
-from modules.shared import opts
+from modules.shared import opts, cmd_opts
 
 LANCZOS = (Image.Resampling.LANCZOS if hasattr(Image, 'Resampling') else Image.LANCZOS)
 
@@ -252,7 +252,7 @@ def sanitize_filename_part(text, replace_spaces=True):
     if replace_spaces:
         text = text.replace(' ', '_')
 
-    return text.translate({ord(x): '' for x in invalid_filename_chars})[:128]
+    return text.translate({ord(x): '_' for x in invalid_filename_chars})[:128]
 
 
 def apply_filename_pattern(x, p, seed, prompt):
@@ -274,16 +274,36 @@ def apply_filename_pattern(x, p, seed, prompt):
         x = x.replace("[height]", str(p.height))
         x = x.replace("[sampler]", sd_samplers.samplers[p.sampler_index].name)
 
-    x = x.replace("[model_hash]", shared.sd_model_hash)
+    x = x.replace("[model_hash]", shared.sd_model.sd_model_hash)
     x = x.replace("[date]", datetime.date.today().isoformat())
 
+    if cmd_opts.hide_ui_dir_config:
+        x = re.sub(r'^[\\/]+|\.{2,}[\\/]+|[\\/]+\.{2,}', '', x)
+
     return x
 
+def get_next_sequence_number(path, basename):
+    """
+    Determines and returns the next sequence number to use when saving an image in the specified directory.
 
-def save_image(image, path, basename, seed=None, prompt=None, extension='png', info=None, short_filename=False, no_prompt=False, pnginfo_section_name='parameters', p=None, existing_info=None):
-    # would be better to add this as an argument in future, but will do for now
-    is_a_grid = basename != ""
+    The sequence starts at 0.
+    """
+    result = -1
+    if basename != '':
+        basename = basename + "-"
 
+    prefix_length = len(basename)
+    for p in os.listdir(path):
+        if p.startswith(basename):
+            l = os.path.splitext(p[prefix_length:])[0].split('-') #splits the filename (removing the basename first if one is defined, so the sequence number is always the first element)
+            try:
+                result = max(int(l[0]), result)
+            except ValueError:
+                pass
+
+    return result + 1
+
+def save_image(image, path, basename, seed=None, prompt=None, extension='png', info=None, short_filename=False, no_prompt=False, grid=False, pnginfo_section_name='parameters', p=None, existing_info=None):
     if short_filename or prompt is None or seed is None:
         file_decoration = ""
     elif opts.save_to_dirs:
@@ -307,7 +327,7 @@ def save_image(image, path, basename, seed=None, prompt=None, extension='png', i
     else:
         pnginfo = None
 
-    save_to_dirs = (is_a_grid and opts.grid_save_to_dirs) or (not is_a_grid and opts.save_to_dirs)
+    save_to_dirs = (grid and opts.grid_save_to_dirs) or (not grid and opts.save_to_dirs and not no_prompt)
 
     if save_to_dirs:
         dirname = apply_filename_pattern(opts.directories_filename_pattern or "[prompt_words]", p, seed, prompt)
@@ -315,26 +335,29 @@ def save_image(image, path, basename, seed=None, prompt=None, extension='png', i
 
     os.makedirs(path, exist_ok=True)
 
-    filecount = len([x for x in os.listdir(path) if os.path.splitext(x)[1] == '.' + extension])
+    basecount = get_next_sequence_number(path, basename)
     fullfn = "a.png"
     fullfn_without_extension = "a"
     for i in range(500):
-        fn = f"{filecount+i:05}" if basename == '' else f"{basename}-{filecount+i:04}"
+        fn = f"{basecount+i:05}" if basename == '' else f"{basename}-{basecount+i:04}"
         fullfn = os.path.join(path, f"{fn}{file_decoration}.{extension}")
         fullfn_without_extension = os.path.join(path, f"{fn}{file_decoration}")
         if not os.path.exists(fullfn):
             break
 
-    if extension.lower() in ("jpg", "jpeg"):
-        exif_bytes = piexif.dump({
+    def exif_bytes():
+        return piexif.dump({
             "Exif": {
-                piexif.ExifIFD.UserComment: info.encode("utf8"),
-            }
+                piexif.ExifIFD.UserComment: piexif.helper.UserComment.dump(info or "", encoding="unicode")
+            },
         })
-    else:
-        exif_bytes = None
 
-    image.save(fullfn, quality=opts.jpeg_quality, pnginfo=pnginfo, exif=exif_bytes)
+    if extension.lower() in ("jpg", "jpeg", "webp"):
+        image.save(fullfn, quality=opts.jpeg_quality)
+        if opts.enable_pnginfo and info is not None:
+            piexif.insert(exif_bytes(), fullfn)
+    else:
+        image.save(fullfn, quality=opts.jpeg_quality, pnginfo=pnginfo)
 
     target_side_length = 4000
     oversize = image.width > target_side_length or image.height > target_side_length
@@ -346,7 +369,9 @@ def save_image(image, path, basename, seed=None, prompt=None, extension='png', i
         elif oversize:
             image = image.resize((image.width * target_side_length // image.height, target_side_length), LANCZOS)
 
-        image.save(fullfn, quality=opts.jpeg_quality, exif=exif_bytes)
+        image.save(fullfn_without_extension + ".jpg", quality=opts.jpeg_quality)
+        if opts.enable_pnginfo and info is not None:
+            piexif.insert(exif_bytes(), fullfn_without_extension + ".jpg")
 
     if opts.save_txt and info is not None:
         with open(f"{fullfn_without_extension}.txt", "w", encoding="utf8") as file:

modules/img2img.py

@@ -11,10 +11,9 @@ from modules.ui import plaintext_to_html
 import modules.images as images
 import modules.scripts
 
-def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init_img_with_mask, init_mask, mask_mode, steps: int, sampler_index: int, mask_blur: int, inpainting_fill: int, restore_faces: bool, tiling: bool, mode: int, n_iter: int, batch_size: int, cfg_scale: float, denoising_strength: float, denoising_strength_change_factor: float, seed: int, subseed: int, subseed_strength: float, seed_resize_from_h: int, seed_resize_from_w: int, height: int, width: int, resize_mode: int, upscaler_index: str, upscale_overlap: int, inpaint_full_res: bool, inpainting_mask_invert: int, *args):
+def img2img(prompt: str, negative_prompt: str, prompt_style: str, prompt_style2: str, init_img, init_img_with_mask, init_mask, mask_mode, steps: int, sampler_index: int, mask_blur: int, inpainting_fill: int, restore_faces: bool, tiling: bool, mode: int, n_iter: int, batch_size: int, cfg_scale: float, denoising_strength: float, seed: int, subseed: int, subseed_strength: float, seed_resize_from_h: int, seed_resize_from_w: int, height: int, width: int, resize_mode: int, upscaler_index: str, upscale_overlap: int, inpaint_full_res: bool, inpainting_mask_invert: int, *args):
     is_inpaint = mode == 1
-    is_loopback = mode == 2
-    is_upscale = mode == 3
+    is_upscale = mode == 2
 
     if is_inpaint:
         if mask_mode == 0:
@@ -38,7 +37,7 @@ def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init
         outpath_grids=opts.outdir_grids or opts.outdir_img2img_grids,
         prompt=prompt,
         negative_prompt=negative_prompt,
-        prompt_style=prompt_style,
+        styles=[prompt_style, prompt_style2],
         seed=seed,
         subseed=subseed,
         subseed_strength=subseed_strength,
@@ -61,46 +60,10 @@ def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init
         denoising_strength=denoising_strength,
         inpaint_full_res=inpaint_full_res,
         inpainting_mask_invert=inpainting_mask_invert,
-        extra_generation_params={
-            "Denoising strength change factor": (denoising_strength_change_factor if is_loopback else None)
-        }
     )
     print(f"\nimg2img: {prompt}", file=shared.progress_print_out)
 
-    if is_loopback:
-        output_images, info = None, None
-        history = []
-        initial_seed = None
-        initial_info = None
-
-        state.job_count = n_iter
-
-        for i in range(n_iter):
-            p.n_iter = 1
-            p.batch_size = 1
-            p.do_not_save_grid = True
-
-            state.job = f"Batch {i + 1} out of {n_iter}"
-            processed = process_images(p)
-
-            if initial_seed is None:
-                initial_seed = processed.seed
-                initial_info = processed.info
-            
-            init_img = processed.images[0]
-
-            p.init_images = [init_img]
-            p.seed = processed.seed + 1
-            p.denoising_strength = min(max(p.denoising_strength * denoising_strength_change_factor, 0.1), 1)
-            history.append(processed.images[0])
-
-        grid = images.image_grid(history, batch_size, rows=1)
-
-        images.save_image(grid, p.outpath_grids, "grid", initial_seed, prompt, opts.grid_format, info=info, short_filename=not opts.grid_extended_filename, p=p)
-
-        processed = Processed(p, history, initial_seed, initial_info)
-
-    elif is_upscale:
+    if is_upscale:
         initial_info = None
 
         processing.fix_seed(p)
@@ -113,6 +76,7 @@ def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init
 
         grid = images.split_grid(img, tile_w=width, tile_h=height, overlap=upscale_overlap)
 
+        batch_size = p.batch_size
         upscale_count = p.n_iter
         p.n_iter = 1
         p.do_not_save_grid = True
@@ -124,7 +88,7 @@ def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init
             for tiledata in row:
                 work.append(tiledata[2])
 
-        batch_count = math.ceil(len(work) / p.batch_size)
+        batch_count = math.ceil(len(work) / batch_size)
         state.job_count = batch_count * upscale_count
 
         print(f"SD upscaling will process a total of {len(work)} images tiled as {len(grid.tiles[0][2])}x{len(grid.tiles)} per upscale in a total of {state.job_count} batches.")
@@ -136,9 +100,10 @@ def img2img(prompt: str, negative_prompt: str, prompt_style: str, init_img, init
 
             work_results = []
             for i in range(batch_count):
-                p.init_images = work[i*p.batch_size:(i+1)*p.batch_size]
+                p.batch_size = batch_size
+                p.init_images = work[i*batch_size:(i+1)*batch_size]
 
-                state.job = f"Batch {i + 1} out of {state.job_count}"
+                state.job = f"Batch {i + 1 + n * batch_count} out of {state.job_count}"
                 processed = process_images(p)
 
                 if initial_info is None:

modules/interrogate.py

@@ -98,7 +98,7 @@ class InterrogateModels:
             text_array = text_array[0:int(shared.opts.interrogate_clip_dict_limit)]
 
         top_count = min(top_count, len(text_array))
-        text_tokens = clip.tokenize([text for text in text_array]).to(shared.device)
+        text_tokens = clip.tokenize([text for text in text_array], truncate=True).to(shared.device)
         text_features = self.clip_model.encode_text(text_tokens).type(self.dtype)
         text_features /= text_features.norm(dim=-1, keepdim=True)
 

modules/masking.py

@@ -0,0 +1,99 @@
+from PIL import Image, ImageFilter, ImageOps
+
+
+def get_crop_region(mask, pad=0):
+    """finds a rectangular region that contains all masked ares in an image. Returns (x1, y1, x2, y2) coordinates of the rectangle.
+    For example, if a user has painted the top-right part of a 512x512 image", the result may be (256, 0, 512, 256)"""
+    
+    h, w = mask.shape
+
+    crop_left = 0
+    for i in range(w):
+        if not (mask[:, i] == 0).all():
+            break
+        crop_left += 1
+
+    crop_right = 0
+    for i in reversed(range(w)):
+        if not (mask[:, i] == 0).all():
+            break
+        crop_right += 1
+
+    crop_top = 0
+    for i in range(h):
+        if not (mask[i] == 0).all():
+            break
+        crop_top += 1
+
+    crop_bottom = 0
+    for i in reversed(range(h)):
+        if not (mask[i] == 0).all():
+            break
+        crop_bottom += 1
+
+    return (
+        int(max(crop_left-pad, 0)),
+        int(max(crop_top-pad, 0)),
+        int(min(w - crop_right + pad, w)),
+        int(min(h - crop_bottom + pad, h))
+    )
+
+
+def expand_crop_region(crop_region, processing_width, processing_height, image_width, image_height):
+    """expands crop region get_crop_region() to match the ratio of the image the region will processed in; returns expanded region
+    for example, if user drew mask in a 128x32 region, and the dimensions for processing are 512x512, the region will be expanded to 128x128."""
+
+    x1, y1, x2, y2 = crop_region
+
+    ratio_crop_region = (x2 - x1) / (y2 - y1)
+    ratio_processing = processing_width / processing_height
+
+    if ratio_crop_region > ratio_processing:
+        desired_height = (x2 - x1) * ratio_processing
+        desired_height_diff = int(desired_height - (y2-y1))
+        y1 -= desired_height_diff//2
+        y2 += desired_height_diff - desired_height_diff//2
+        if y2 >= image_height:
+            diff = y2 - image_height
+            y2 -= diff
+            y1 -= diff
+        if y1 < 0:
+            y2 -= y1
+            y1 -= y1
+        if y2 >= image_height:
+            y2 = image_height
+    else:
+        desired_width = (y2 - y1) * ratio_processing
+        desired_width_diff = int(desired_width - (x2-x1))
+        x1 -= desired_width_diff//2
+        x2 += desired_width_diff - desired_width_diff//2
+        if x2 >= image_width:
+            diff = x2 - image_width
+            x2 -= diff
+            x1 -= diff
+        if x1 < 0:
+            x2 -= x1
+            x1 -= x1
+        if x2 >= image_width:
+            x2 = image_width
+
+    return x1, y1, x2, y2
+
+
+def fill(image, mask):
+    """fills masked regions with colors from image using blur. Not extremely effective."""
+
+    image_mod = Image.new('RGBA', (image.width, image.height))
+
+    image_masked = Image.new('RGBa', (image.width, image.height))
+    image_masked.paste(image.convert("RGBA").convert("RGBa"), mask=ImageOps.invert(mask.convert('L')))
+
+    image_masked = image_masked.convert('RGBa')
+
+    for radius, repeats in [(256, 1), (64, 1), (16, 2), (4, 4), (2, 2), (0, 1)]:
+        blurred = image_masked.filter(ImageFilter.GaussianBlur(radius)).convert('RGBA')
+        for _ in range(repeats):
+            image_mod.alpha_composite(blurred)
+
+    return image_mod.convert("RGB")
+

modules/memmon.py

@@ -0,0 +1,85 @@
+import threading
+import time
+from collections import defaultdict
+
+import torch
+
+
+class MemUsageMonitor(threading.Thread):
+    run_flag = None
+    device = None
+    disabled = False
+    opts = None
+    data = None
+
+    def __init__(self, name, device, opts):
+        threading.Thread.__init__(self)
+        self.name = name
+        self.device = device
+        self.opts = opts
+
+        self.daemon = True
+        self.run_flag = threading.Event()
+        self.data = defaultdict(int)
+
+        try:
+            torch.cuda.mem_get_info()
+            torch.cuda.memory_stats(self.device)
+        except Exception as e:  # AMD or whatever
+            print(f"Warning: caught exception '{e}', memory monitor disabled")
+            self.disabled = True
+
+    def run(self):
+        if self.disabled:
+            return
+
+        while True:
+            self.run_flag.wait()
+
+            torch.cuda.reset_peak_memory_stats()
+            self.data.clear()
+
+            if self.opts.memmon_poll_rate <= 0:
+                self.run_flag.clear()
+                continue
+
+            self.data["min_free"] = torch.cuda.mem_get_info()[0]
+
+            while self.run_flag.is_set():
+                free, total = torch.cuda.mem_get_info()  # calling with self.device errors, torch bug?
+                self.data["min_free"] = min(self.data["min_free"], free)
+
+                time.sleep(1 / self.opts.memmon_poll_rate)
+
+    def dump_debug(self):
+        print(self, 'recorded data:')
+        for k, v in self.read().items():
+            print(k, -(v // -(1024 ** 2)))
+
+        print(self, 'raw torch memory stats:')
+        tm = torch.cuda.memory_stats(self.device)
+        for k, v in tm.items():
+            if 'bytes' not in k:
+                continue
+            print('\t' if 'peak' in k else '', k, -(v // -(1024 ** 2)))
+
+        print(torch.cuda.memory_summary())
+
+    def monitor(self):
+        self.run_flag.set()
+
+    def read(self):
+        if not self.disabled:
+            free, total = torch.cuda.mem_get_info()
+            self.data["total"] = total
+
+            torch_stats = torch.cuda.memory_stats(self.device)
+            self.data["active_peak"] = torch_stats["active_bytes.all.peak"]
+            self.data["reserved_peak"] = torch_stats["reserved_bytes.all.peak"]
+            self.data["system_peak"] = total - self.data["min_free"]
+
+        return self.data
+
+    def stop(self):
+        self.run_flag.clear()
+        return self.read()

modules/processing.py

@@ -12,7 +12,7 @@ import cv2
 from skimage import exposure
 
 import modules.sd_hijack
-from modules import devices
+from modules import devices, prompt_parser, masking
 from modules.sd_hijack import model_hijack
 from modules.sd_samplers import samplers, samplers_for_img2img
 from modules.shared import opts, cmd_opts, state
@@ -46,14 +46,14 @@ def apply_color_correction(correction, image):
 
 
 class StableDiffusionProcessing:
-    def __init__(self, sd_model=None, outpath_samples=None, outpath_grids=None, prompt="", prompt_style="None", seed=-1, subseed=-1, subseed_strength=0, seed_resize_from_h=-1, seed_resize_from_w=-1, sampler_index=0, batch_size=1, n_iter=1, steps=50, cfg_scale=7.0, width=512, height=512, restore_faces=False, tiling=False, do_not_save_samples=False, do_not_save_grid=False, extra_generation_params=None, overlay_images=None, negative_prompt=None):
+    def __init__(self, sd_model=None, outpath_samples=None, outpath_grids=None, prompt="", styles=None, seed=-1, subseed=-1, subseed_strength=0, seed_resize_from_h=-1, seed_resize_from_w=-1, sampler_index=0, batch_size=1, n_iter=1, steps=50, cfg_scale=7.0, width=512, height=512, restore_faces=False, tiling=False, do_not_save_samples=False, do_not_save_grid=False, extra_generation_params=None, overlay_images=None, negative_prompt=None):
         self.sd_model = sd_model
         self.outpath_samples: str = outpath_samples
         self.outpath_grids: str = outpath_grids
         self.prompt: str = prompt
         self.prompt_for_display: str = None
         self.negative_prompt: str = (negative_prompt or "")
-        self.prompt_style: str = prompt_style
+        self.styles: str = styles
         self.seed: int = seed
         self.subseed: int = subseed
         self.subseed_strength: float = subseed_strength
@@ -74,46 +74,92 @@ class StableDiffusionProcessing:
         self.overlay_images = overlay_images
         self.paste_to = None
         self.color_corrections = None
+        self.denoising_strength: float = 0
 
-    def init(self, seed):
+    def init(self, all_prompts, all_seeds, all_subseeds):
         pass
 
-    def sample(self, x, conditioning, unconditional_conditioning):
+    def sample(self, conditioning, unconditional_conditioning, seeds, subseeds, subseed_strength):
         raise NotImplementedError()
 
 
 class Processed:
-    def __init__(self, p: StableDiffusionProcessing, images_list, seed, info):
+    def __init__(self, p: StableDiffusionProcessing, images_list, seed=-1, info="", subseed=None, all_prompts=None, all_seeds=None, all_subseeds=None, index_of_first_image=0):
         self.images = images_list
         self.prompt = p.prompt
         self.negative_prompt = p.negative_prompt
         self.seed = seed
+        self.subseed = subseed
+        self.subseed_strength = p.subseed_strength
         self.info = info
         self.width = p.width
         self.height = p.height
+        self.sampler_index = p.sampler_index
         self.sampler = samplers[p.sampler_index].name
         self.cfg_scale = p.cfg_scale
         self.steps = p.steps
+        self.batch_size = p.batch_size
+        self.restore_faces = p.restore_faces
+        self.face_restoration_model = opts.face_restoration_model if p.restore_faces else None
+        self.sd_model_hash = shared.sd_model.sd_model_hash
+        self.seed_resize_from_w = p.seed_resize_from_w
+        self.seed_resize_from_h = p.seed_resize_from_h
+        self.denoising_strength = getattr(p, 'denoising_strength', None)
+        self.extra_generation_params = p.extra_generation_params
+        self.index_of_first_image = index_of_first_image
+
+        self.prompt = self.prompt if type(self.prompt) != list else self.prompt[0]
+        self.negative_prompt = self.negative_prompt if type(self.negative_prompt) != list else self.negative_prompt[0]
+        self.seed = int(self.seed if type(self.seed) != list else self.seed[0])
+        self.subseed = int(self.subseed if type(self.subseed) != list else self.subseed[0]) if self.subseed is not None else -1
+
+        self.all_prompts = all_prompts or [self.prompt]
+        self.all_seeds = all_seeds or [self.seed]
+        self.all_subseeds = all_subseeds or [self.subseed]
 
     def js(self):
         obj = {
-            "prompt": self.prompt if type(self.prompt) != list else self.prompt[0],
-            "negative_prompt": self.negative_prompt if type(self.negative_prompt) != list else self.negative_prompt[0],
-            "seed": int(self.seed if type(self.seed) != list else self.seed[0]),
+            "prompt": self.prompt,
+            "all_prompts": self.all_prompts,
+            "negative_prompt": self.negative_prompt,
+            "seed": self.seed,
+            "all_seeds": self.all_seeds,
+            "subseed": self.subseed,
+            "all_subseeds": self.all_subseeds,
+            "subseed_strength": self.subseed_strength,
             "width": self.width,
             "height": self.height,
+            "sampler_index": self.sampler_index,
             "sampler": self.sampler,
             "cfg_scale": self.cfg_scale,
             "steps": self.steps,
+            "batch_size": self.batch_size,
+            "restore_faces": self.restore_faces,
+            "face_restoration_model": self.face_restoration_model,
+            "sd_model_hash": self.sd_model_hash,
+            "seed_resize_from_w": self.seed_resize_from_w,
+            "seed_resize_from_h": self.seed_resize_from_h,
+            "denoising_strength": self.denoising_strength,
+            "extra_generation_params": self.extra_generation_params,
+            "index_of_first_image": self.index_of_first_image,
         }
 
         return json.dumps(obj)
 
+    def infotext(self,  p: StableDiffusionProcessing, index):
+        return create_infotext(p, self.all_prompts, self.all_seeds, self.all_subseeds, comments=[], position_in_batch=index % self.batch_size, iteration=index // self.batch_size)
+
+
 # from https://discuss.pytorch.org/t/help-regarding-slerp-function-for-generative-model-sampling/32475/3
 def slerp(val, low, high):
     low_norm = low/torch.norm(low, dim=1, keepdim=True)
     high_norm = high/torch.norm(high, dim=1, keepdim=True)
-    omega = torch.acos((low_norm*high_norm).sum(1))
+    dot = (low_norm*high_norm).sum(1)
+
+    if dot.mean() > 0.9995:
+        return low * val + high * (1 - val)
+
+    omega = torch.acos(dot)
     so = torch.sin(omega)
     res = (torch.sin((1.0-val)*omega)/so).unsqueeze(1)*low + (torch.sin(val*omega)/so).unsqueeze(1) * high
     return res
@@ -122,6 +168,10 @@ def slerp(val, low, high):
 def create_random_tensors(shape, seeds, subseeds=None, subseed_strength=0.0, seed_resize_from_h=0, seed_resize_from_w=0, p=None):
     xs = []
 
+    # if we have multiple seeds, this means we are working with batch size>1; this then
+    # enables the generation of additional tensors with noise that the sampler will use during its processing.
+    # Using those pre-generated tensors instead of simple torch.randn allows a batch with seeds [100, 101] to
+    # produce the same images as with two batches [100], [101].
     if p is not None and p.sampler is not None and len(seeds) > 1 and opts.enable_batch_seeds:
         sampler_noises = [[] for _ in range(p.sampler.number_of_needed_noises(p))]
     else:
@@ -143,11 +193,9 @@ def create_random_tensors(shape, seeds, subseeds=None, subseed_strength=0.0, see
         noise = devices.randn(seed, noise_shape)
 
         if subnoise is not None:
-            #noise = subnoise * subseed_strength + noise * (1 - subseed_strength)
             noise = slerp(subseed_strength, noise, subnoise)
 
         if noise_shape != shape:
-            #noise = torch.nn.functional.interpolate(noise.unsqueeze(1), size=shape[1:], mode="bilinear").squeeze()
             x = devices.randn(seed, shape)
             dx = (shape[2] - noise_shape[2]) // 2
             dy = (shape[1] - noise_shape[1]) // 2
@@ -181,10 +229,43 @@ def fix_seed(p):
     p.subseed = int(random.randrange(4294967294)) if p.subseed is None or p.subseed == -1 else p.subseed
 
 
+def create_infotext(p, all_prompts, all_seeds, all_subseeds, comments, iteration=0, position_in_batch=0):
+    index = position_in_batch + iteration * p.batch_size
+
+    generation_params = {
+        "Steps": p.steps,
+        "Sampler": samplers[p.sampler_index].name,
+        "CFG scale": p.cfg_scale,
+        "Seed": all_seeds[index],
+        "Face restoration": (opts.face_restoration_model if p.restore_faces else None),
+        "Size": f"{p.width}x{p.height}",
+        "Model hash": getattr(p, 'sd_model_hash', None if not opts.add_model_hash_to_info or not shared.sd_model.sd_model_hash else shared.sd_model.sd_model_hash),
+        "Batch size": (None if p.batch_size < 2 else p.batch_size),
+        "Batch pos": (None if p.batch_size < 2 else position_in_batch),
+        "Variation seed": (None if p.subseed_strength == 0 else all_subseeds[index]),
+        "Variation seed strength": (None if p.subseed_strength == 0 else p.subseed_strength),
+        "Seed resize from": (None if p.seed_resize_from_w == 0 or p.seed_resize_from_h == 0 else f"{p.seed_resize_from_w}x{p.seed_resize_from_h}"),
+        "Denoising strength": getattr(p, 'denoising_strength', None),
+    }
+
+    if p.extra_generation_params is not None:
+        generation_params.update(p.extra_generation_params)
+
+    generation_params_text = ", ".join([k if k == v else f'{k}: {v}' for k, v in generation_params.items() if v is not None])
+
+    negative_prompt_text = "\nNegative prompt: " + p.negative_prompt if p.negative_prompt else ""
+
+    return f"{all_prompts[index]}{negative_prompt_text}\n{generation_params_text}".strip() + "".join(["\n\n" + x for x in comments])
+
+
 def process_images(p: StableDiffusionProcessing) -> Processed:
     """this is the main loop that both txt2img and img2img use; it calls func_init once inside all the scopes and func_sample once per batch"""
 
-    assert p.prompt is not None
+    if type(p.prompt) == list:
+        assert(len(p.prompt) > 0)
+    else:
+        assert p.prompt is not None
+        
     devices.torch_gc()
 
     fix_seed(p)
@@ -194,9 +275,9 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
 
     modules.sd_hijack.model_hijack.apply_circular(p.tiling)
 
-    comments = []
+    comments = {}
 
-    modules.styles.apply_style(p, shared.prompt_styles[p.prompt_style])
+    shared.prompt_styles.apply_styles(p)
 
     if type(p.prompt) == list:
         all_prompts = p.prompt
@@ -214,32 +295,7 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
         all_subseeds = [int(p.subseed + x) for x in range(len(all_prompts))]
 
     def infotext(iteration=0, position_in_batch=0):
-        index = position_in_batch + iteration * p.batch_size
-
-        generation_params = {
-            "Steps": p.steps,
-            "Sampler": samplers[p.sampler_index].name,
-            "CFG scale": p.cfg_scale,
-            "Seed": all_seeds[index],
-            "Face restoration": (opts.face_restoration_model if p.restore_faces else None),
-            "Size": f"{p.width}x{p.height}",
-            "Model hash": (None if not opts.add_model_hash_to_info or not shared.sd_model_hash else shared.sd_model_hash),
-            "Batch size": (None if p.batch_size < 2 else p.batch_size),
-            "Batch pos": (None if p.batch_size < 2 else position_in_batch),
-            "Variation seed": (None if p.subseed_strength == 0 else all_subseeds[index]),
-            "Variation seed strength": (None if p.subseed_strength == 0 else p.subseed_strength),
-            "Seed resize from": (None if p.seed_resize_from_w == 0 or p.seed_resize_from_h == 0 else f"{p.seed_resize_from_w}x{p.seed_resize_from_h}"),
-            "Denoising strength": getattr(p, 'denoising_strength', None),
-        }
-
-        if p.extra_generation_params is not None:
-            generation_params.update(p.extra_generation_params)
-
-        generation_params_text = ", ".join([k if k == v else f'{k}: {v}' for k, v in generation_params.items() if v is not None])
-        
-        negative_prompt_text = "\nNegative prompt: " + p.negative_prompt if p.negative_prompt else ""
-
-        return f"{all_prompts[index]}{negative_prompt_text}\n{generation_params_text}".strip() + "".join(["\n\n" + x for x in comments])
+        return create_infotext(p, all_prompts, all_seeds, all_subseeds, comments, iteration, position_in_batch)
 
     if os.path.exists(cmd_opts.embeddings_dir):
         model_hijack.load_textual_inversion_embeddings(cmd_opts.embeddings_dir, p.sd_model)
@@ -248,7 +304,7 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
     precision_scope = torch.autocast if cmd_opts.precision == "autocast" else contextlib.nullcontext
     ema_scope = (contextlib.nullcontext if cmd_opts.lowvram else p.sd_model.ema_scope)
     with torch.no_grad(), precision_scope("cuda"), ema_scope():
-        p.init(seed=all_seeds[0])
+        p.init(all_prompts, all_seeds, all_subseeds)
 
         if state.job_count == -1:
             state.job_count = p.n_iter
@@ -261,19 +317,22 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
             seeds = all_seeds[n * p.batch_size:(n + 1) * p.batch_size]
             subseeds = all_subseeds[n * p.batch_size:(n + 1) * p.batch_size]
 
-            uc = p.sd_model.get_learned_conditioning(len(prompts) * [p.negative_prompt])
-            c = p.sd_model.get_learned_conditioning(prompts)
+            if (len(prompts) == 0):
+                break
+
+            #uc = p.sd_model.get_learned_conditioning(len(prompts) * [p.negative_prompt])
+            #c = p.sd_model.get_learned_conditioning(prompts)
+            uc = prompt_parser.get_learned_conditioning(len(prompts) * [p.negative_prompt], p.steps)
+            c = prompt_parser.get_learned_conditioning(prompts, p.steps)
 
             if len(model_hijack.comments) > 0:
-                comments += model_hijack.comments
-
-            # we manually generate all input noises because each one should have a specific seed
-            x = create_random_tensors([opt_C, p.height // opt_f, p.width // opt_f], seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, seed_resize_from_h=p.seed_resize_from_h, seed_resize_from_w=p.seed_resize_from_w, p=p)
+                for comment in model_hijack.comments:
+                    comments[comment] = 1
 
             if p.n_iter > 1:
                 shared.state.job = f"Batch {n+1} out of {p.n_iter}"
 
-            samples_ddim = p.sample(x=x, conditioning=c, unconditional_conditioning=uc)
+            samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength)
             if state.interrupted:
 
                 # if we are interruped, sample returns just noise
@@ -325,82 +384,83 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
 
             state.nextjob()
 
-        unwanted_grid_because_of_img_count = len(output_images) < 2 and opts.grid_only_if_multiple
-        if not p.do_not_save_grid and not unwanted_grid_because_of_img_count:
-            return_grid = opts.return_grid
+        p.color_corrections = None
 
+        index_of_first_image = 0
+        unwanted_grid_because_of_img_count = len(output_images) < 2 and opts.grid_only_if_multiple
+        if (opts.return_grid or opts.grid_save) and not p.do_not_save_grid and not unwanted_grid_because_of_img_count:
             grid = images.image_grid(output_images, p.batch_size)
 
-            if return_grid:
+            if opts.return_grid:
                 output_images.insert(0, grid)
+                index_of_first_image = 1
 
             if opts.grid_save:
                 images.save_image(grid, p.outpath_grids, "grid", all_seeds[0], all_prompts[0], opts.grid_format, info=infotext(), short_filename=not opts.grid_extended_filename, p=p)
 
     devices.torch_gc()
-    return Processed(p, output_images, all_seeds[0], infotext())
+    return Processed(p, output_images, all_seeds[0], infotext(), subseed=all_subseeds[0], all_prompts=all_prompts, all_seeds=all_seeds, all_subseeds=all_subseeds, index_of_first_image=index_of_first_image)
 
 
 class StableDiffusionProcessingTxt2Img(StableDiffusionProcessing):
     sampler = None
+    firstphase_width = 0
+    firstphase_height = 0
+    firstphase_width_truncated = 0
+    firstphase_height_truncated = 0
 
-    def init(self, seed):
+    def __init__(self, enable_hr=False, scale_latent=True, denoising_strength=0.75, **kwargs):
+        super().__init__(**kwargs)
+        self.enable_hr = enable_hr
+        self.scale_latent = scale_latent
+        self.denoising_strength = denoising_strength
+
+    def init(self, all_prompts, all_seeds, all_subseeds):
+        if self.enable_hr:
+            if state.job_count == -1:
+                state.job_count = self.n_iter * 2
+            else:
+                state.job_count = state.job_count * 2
+
+            desired_pixel_count = 512 * 512
+            actual_pixel_count = self.width * self.height
+            scale = math.sqrt(desired_pixel_count / actual_pixel_count)
+
+            self.firstphase_width = math.ceil(scale * self.width / 64) * 64
+            self.firstphase_height = math.ceil(scale * self.height / 64) * 64
+            self.firstphase_width_truncated = int(scale * self.width)
+            self.firstphase_height_truncated = int(scale * self.height)
+
+    def sample(self, conditioning, unconditional_conditioning, seeds, subseeds, subseed_strength):
         self.sampler = samplers[self.sampler_index].constructor(self.sd_model)
 
-    def sample(self, x, conditioning, unconditional_conditioning):
-        samples_ddim = self.sampler.sample(self, x, conditioning, unconditional_conditioning)
-        return samples_ddim
+        if not self.enable_hr:
+            x = create_random_tensors([opt_C, self.height // opt_f, self.width // opt_f], seeds=seeds, subseeds=subseeds, subseed_strength=self.subseed_strength, seed_resize_from_h=self.seed_resize_from_h, seed_resize_from_w=self.seed_resize_from_w, p=self)
+            samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning)
+            return samples
 
+        x = create_random_tensors([opt_C, self.firstphase_height // opt_f, self.firstphase_width // opt_f], seeds=seeds, subseeds=subseeds, subseed_strength=self.subseed_strength, seed_resize_from_h=self.seed_resize_from_h, seed_resize_from_w=self.seed_resize_from_w, p=self)
+        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning)
 
-def get_crop_region(mask, pad=0):
-    h, w = mask.shape
+        truncate_x = (self.firstphase_width - self.firstphase_width_truncated) // opt_f
+        truncate_y = (self.firstphase_height - self.firstphase_height_truncated) // opt_f
 
-    crop_left = 0
-    for i in range(w):
-        if not (mask[:, i] == 0).all():
-            break
-        crop_left += 1
+        samples = samples[:, :, truncate_y//2:samples.shape[2]-truncate_y//2, truncate_x//2:samples.shape[3]-truncate_x//2]
 
-    crop_right = 0
-    for i in reversed(range(w)):
-        if not (mask[:, i] == 0).all():
-            break
-        crop_right += 1
+        if self.scale_latent:
+            samples = torch.nn.functional.interpolate(samples, size=(self.height // opt_f, self.width // opt_f), mode="bilinear")
+        else:
+            decoded_samples = self.sd_model.decode_first_stage(samples)
+            decoded_samples = torch.nn.functional.interpolate(decoded_samples, size=(self.height, self.width), mode="bilinear")
+            samples = self.sd_model.get_first_stage_encoding(self.sd_model.encode_first_stage(decoded_samples))
 
-    crop_top = 0
-    for i in range(h):
-        if not (mask[i] == 0).all():
-            break
-        crop_top += 1
+        shared.state.nextjob()
 
-    crop_bottom = 0
-    for i in reversed(range(h)):
-        if not (mask[i] == 0).all():
-            break
-        crop_bottom += 1
+        self.sampler = samplers[self.sampler_index].constructor(self.sd_model)
+        noise = create_random_tensors(samples.shape[1:], seeds=seeds, subseeds=subseeds, subseed_strength=subseed_strength, seed_resize_from_h=self.seed_resize_from_h, seed_resize_from_w=self.seed_resize_from_w, p=self)
+        samples = self.sampler.sample_img2img(self, samples, noise, conditioning, unconditional_conditioning, steps=self.steps)
 
-    return (
-        int(max(crop_left-pad, 0)),
-        int(max(crop_top-pad, 0)),
-        int(min(w - crop_right + pad, w)),
-        int(min(h - crop_bottom + pad, h))
-    )
-
-
-def fill(image, mask):
-    image_mod = Image.new('RGBA', (image.width, image.height))
-
-    image_masked = Image.new('RGBa', (image.width, image.height))
-    image_masked.paste(image.convert("RGBA").convert("RGBa"), mask=ImageOps.invert(mask.convert('L')))
-
-    image_masked = image_masked.convert('RGBa')
-
-    for radius, repeats in [(256, 1), (64, 1), (16, 2), (4, 4), (2, 2), (0, 1)]:
-        blurred = image_masked.filter(ImageFilter.GaussianBlur(radius)).convert('RGBA')
-        for _ in range(repeats):
-            image_mod.alpha_composite(blurred)
-
-    return image_mod.convert("RGB")
+        return samples
 
 
 class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
@@ -424,7 +484,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
         self.mask = None
         self.nmask = None
 
-    def init(self, seed):
+    def init(self, all_prompts, all_seeds, all_subseeds):
         self.sampler = samplers_for_img2img[self.sampler_index].constructor(self.sd_model)
         crop_region = None
 
@@ -442,7 +502,8 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
             if self.inpaint_full_res:
                 self.mask_for_overlay = self.image_mask
                 mask = self.image_mask.convert('L')
-                crop_region = get_crop_region(np.array(mask), opts.upscale_at_full_resolution_padding)
+                crop_region = masking.get_crop_region(np.array(mask), opts.upscale_at_full_resolution_padding)
+                crop_region = masking.expand_crop_region(crop_region, self.width, self.height, mask.width, mask.height)
                 x1, y1, x2, y2 = crop_region
 
                 mask = mask.crop(crop_region)
@@ -458,7 +519,9 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
 
         latent_mask = self.latent_mask if self.latent_mask is not None else self.image_mask
 
-        self.color_corrections = []
+        add_color_corrections = opts.img2img_color_correction and self.color_corrections is None
+        if add_color_corrections:
+            self.color_corrections = []
         imgs = []
         for img in self.init_images:
             image = img.convert("RGB")
@@ -478,9 +541,9 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
 
             if self.image_mask is not None:
                 if self.inpainting_fill != 1:
-                    image = fill(image, latent_mask)
+                    image = masking.fill(image, latent_mask)
 
-            if opts.img2img_color_correction:
+            if add_color_corrections:
                 self.color_corrections.append(setup_color_correction(image))
 
             image = np.array(image).astype(np.float32) / 255.0
@@ -515,12 +578,15 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
             self.mask = torch.asarray(1.0 - latmask).to(shared.device).type(self.sd_model.dtype)
             self.nmask = torch.asarray(latmask).to(shared.device).type(self.sd_model.dtype)
 
+            # this needs to be fixed to be done in sample() using actual seeds for batches
             if self.inpainting_fill == 2:
-                self.init_latent = self.init_latent * self.mask + create_random_tensors(self.init_latent.shape[1:], [seed + x + 1 for x in range(self.init_latent.shape[0])]) * self.nmask
+                self.init_latent = self.init_latent * self.mask + create_random_tensors(self.init_latent.shape[1:], all_seeds[0:self.init_latent.shape[0]]) * self.nmask
             elif self.inpainting_fill == 3:
                 self.init_latent = self.init_latent * self.mask
 
-    def sample(self, x, conditioning, unconditional_conditioning):
+    def sample(self, conditioning, unconditional_conditioning, seeds, subseeds, subseed_strength):
+        x = create_random_tensors([opt_C, self.height // opt_f, self.width // opt_f], seeds=seeds, subseeds=subseeds, subseed_strength=self.subseed_strength, seed_resize_from_h=self.seed_resize_from_h, seed_resize_from_w=self.seed_resize_from_w, p=self)
+
         samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning)
 
         if self.mask is not None:

modules/prompt_parser.py

@@ -0,0 +1,130 @@
+import re
+from collections import namedtuple
+import torch
+
+import modules.shared as shared
+
+re_prompt = re.compile(r'''
+(.*?)
+\[
+    ([^]:]+):
+    (?:([^]:]*):)?
+    ([0-9]*\.?[0-9]+)
+]
+|
+(.+)
+''', re.X)
+
+# a prompt like this: "fantasy landscape with a [mountain:lake:0.25] and [an oak:a christmas tree:0.75][ in foreground::0.6][ in background:0.25] [shoddy:masterful:0.5]"
+# will be represented with prompt_schedule like this (assuming steps=100):
+# [25, 'fantasy landscape with a mountain and an oak in foreground shoddy']
+# [50, 'fantasy landscape with a lake and an oak in foreground in background shoddy']
+# [60, 'fantasy landscape with a lake and an oak in foreground in background masterful']
+# [75, 'fantasy landscape with a lake and an oak in background masterful']
+# [100, 'fantasy landscape with a lake and a christmas tree in background masterful']
+
+
+def get_learned_conditioning_prompt_schedules(prompts, steps):
+    res = []
+    cache = {}
+
+    for prompt in prompts:
+        prompt_schedule: list[list[str | int]] = [[steps, ""]]
+
+        cached = cache.get(prompt, None)
+        if cached is not None:
+            res.append(cached)
+            continue
+
+        for m in re_prompt.finditer(prompt):
+            plaintext = m.group(1) if m.group(5) is None else m.group(5)
+            concept_from = m.group(2)
+            concept_to = m.group(3)
+            if concept_to is None:
+                concept_to = concept_from
+                concept_from = ""
+            swap_position = float(m.group(4)) if m.group(4) is not None else None
+
+            if swap_position is not None:
+                if swap_position < 1:
+                    swap_position = swap_position * steps
+                swap_position = int(min(swap_position, steps))
+
+            swap_index = None
+            found_exact_index = False
+            for i in range(len(prompt_schedule)):
+                end_step = prompt_schedule[i][0]
+                prompt_schedule[i][1] += plaintext
+
+                if swap_position is not None and swap_index is None:
+                    if swap_position == end_step:
+                        swap_index = i
+                        found_exact_index = True
+
+                    if swap_position < end_step:
+                        swap_index = i
+
+            if swap_index is not None:
+                if not found_exact_index:
+                    prompt_schedule.insert(swap_index, [swap_position, prompt_schedule[swap_index][1]])
+
+                for i in range(len(prompt_schedule)):
+                    end_step = prompt_schedule[i][0]
+                    must_replace = swap_position < end_step
+
+                    prompt_schedule[i][1] += concept_to if must_replace else concept_from
+
+        res.append(prompt_schedule)
+        cache[prompt] = prompt_schedule
+        #for t in prompt_schedule:
+        #    print(t)
+
+    return res
+
+
+ScheduledPromptConditioning = namedtuple("ScheduledPromptConditioning", ["end_at_step", "cond"])
+ScheduledPromptBatch = namedtuple("ScheduledPromptBatch", ["shape", "schedules"])
+
+
+def get_learned_conditioning(prompts, steps):
+
+    res = []
+
+    prompt_schedules = get_learned_conditioning_prompt_schedules(prompts, steps)
+    cache = {}
+
+    for prompt, prompt_schedule in zip(prompts, prompt_schedules):
+
+        cached = cache.get(prompt, None)
+        if cached is not None:
+            res.append(cached)
+            continue
+
+        texts = [x[1] for x in prompt_schedule]
+        conds = shared.sd_model.get_learned_conditioning(texts)
+
+        cond_schedule = []
+        for i, (end_at_step, text) in enumerate(prompt_schedule):
+            cond_schedule.append(ScheduledPromptConditioning(end_at_step, conds[i]))
+
+        cache[prompt] = cond_schedule
+        res.append(cond_schedule)
+
+    return ScheduledPromptBatch((len(prompts),) + res[0][0].cond.shape, res)
+
+
+def reconstruct_cond_batch(c: ScheduledPromptBatch, current_step):
+    res = torch.zeros(c.shape)
+    for i, cond_schedule in enumerate(c.schedules):
+        target_index = 0
+        for curret_index, (end_at, cond) in enumerate(cond_schedule):
+            if current_step <= end_at:
+                target_index = curret_index
+                break
+        res[i] = cond_schedule[target_index].cond
+
+    return res.to(shared.device)
+
+
+
+#get_learned_conditioning_prompt_schedules(["fantasy landscape with a [mountain:lake:0.25] and [an oak:a christmas tree:0.75][ in foreground::0.6][ in background:0.25] [shoddy:masterful:0.5]"], 100)

modules/scripts.py

@@ -13,18 +13,37 @@ class Script:
     args_from = None
     args_to = None
 
+    # The title of the script. This is what will be displayed in the dropdown menu.
     def title(self):
         raise NotImplementedError()
 
+    # How the script is displayed in the UI. See https://gradio.app/docs/#components
+    # for the different UI components you can use and how to create them.
+    # Most UI components can return a value, such as a boolean for a checkbox.
+    # The returned values are passed to the run method as parameters.
     def ui(self, is_img2img):
         pass
 
+    # Determines when the script should be shown in the dropdown menu via the 
+    # returned value. As an example:
+    # is_img2img is True if the current tab is img2img, and False if it is txt2img.
+    # Thus, return is_img2img to only show the script on the img2img tab.
     def show(self, is_img2img):
         return True
 
+    # This is where the additional processing is implemented. The parameters include
+    # self, the model object "p" (a StableDiffusionProcessing class, see
+    # processing.py), and the parameters returned by the ui method.
+    # Custom functions can be defined here, and additional libraries can be imported 
+    # to be used in processing. The return value should be a Processed object, which is
+    # what is returned by the process_images method.
     def run(self, *args):
         raise NotImplementedError()
 
+    # The description method is currently unused.
+    # To add a description that appears when hovering over the title, amend the "titles" 
+    # dict in script.js to include the script title (returned by title) as a key, and 
+    # your description as the value.
     def describe(self):
         return ""
 
@@ -42,10 +61,10 @@ def load_scripts(basedir):
         if not os.path.isfile(path):
             continue
 
-        with open(path, "r", encoding="utf8") as file:
-            text = file.read()
-
         try:
+            with open(path, "r", encoding="utf8") as file:
+                text = file.read()
+
             from types import ModuleType
             compiled = compile(text, path, 'exec')
             module = ModuleType(filename)
@@ -92,6 +111,7 @@ class ScriptRunner:
 
         for script in self.scripts:
             script.args_from = len(inputs)
+            script.args_to = len(inputs)
 
             controls = wrap_call(script.ui, script.filename, "ui", is_img2img)
 

modules/sd_hijack.py

@@ -50,14 +50,14 @@ def split_cross_attention_forward(self, x, context=None, mask=None):
 
     q_in = self.to_q(x)
     context = default(context, x)
-    k_in = self.to_k(context)
+    k_in = self.to_k(context) * self.scale
     v_in = self.to_v(context)
     del context, x
 
     q, k, v = map(lambda t: rearrange(t, 'b n (h d) -> (b h) n d', h=h), (q_in, k_in, v_in))
     del q_in, k_in, v_in
 
-    r1 = torch.zeros(q.shape[0], q.shape[1], v.shape[2], device=q.device)
+    r1 = torch.zeros(q.shape[0], q.shape[1], v.shape[2], device=q.device, dtype=q.dtype)
 
     stats = torch.cuda.memory_stats(q.device)
     mem_active = stats['active_bytes.all.current']
@@ -85,7 +85,7 @@ def split_cross_attention_forward(self, x, context=None, mask=None):
     slice_size = q.shape[1] // steps if (q.shape[1] % steps) == 0 else q.shape[1]
     for i in range(0, q.shape[1], slice_size):
         end = i + slice_size
-        s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k) * self.scale
+        s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
 
         s2 = s1.softmax(dim=-1, dtype=q.dtype)
         del s1
@@ -243,12 +243,12 @@ class StableDiffusionModelHijack:
         model_embeddings.token_embedding = EmbeddingsWithFixes(model_embeddings.token_embedding, self)
         m.cond_stage_model = FrozenCLIPEmbedderWithCustomWords(m.cond_stage_model, self)
 
-        if cmd_opts.opt_split_attention:
+        if cmd_opts.opt_split_attention_v1:
+            ldm.modules.attention.CrossAttention.forward = split_cross_attention_forward_v1
+        elif not cmd_opts.disable_opt_split_attention:
             ldm.modules.attention.CrossAttention.forward = split_cross_attention_forward
             ldm.modules.diffusionmodules.model.nonlinearity = nonlinearity_hijack
             ldm.modules.diffusionmodules.model.AttnBlock.forward = cross_attention_attnblock_forward
-        elif cmd_opts.opt_split_attention_v1:
-            ldm.modules.attention.CrossAttention.forward = split_cross_attention_forward_v1
 
         def flatten(el):
             flattened = [flatten(children) for children in el.children()]

modules/sd_models.py

@@ -0,0 +1,151 @@
+import glob
+import os.path
+import sys
+from collections import namedtuple
+import torch
+from omegaconf import OmegaConf
+
+
+from ldm.util import instantiate_from_config
+
+from modules import shared
+
+CheckpointInfo = namedtuple("CheckpointInfo", ['filename', 'title', 'hash'])
+checkpoints_list = {}
+
+try:
+    # this silences the annoying "Some weights of the model checkpoint were not used when initializing..." message at start.
+
+    from transformers import logging
+
+    logging.set_verbosity_error()
+except Exception:
+    pass
+
+
+def list_models():
+    checkpoints_list.clear()
+
+    model_dir = os.path.abspath(shared.cmd_opts.ckpt_dir)
+
+    def modeltitle(path, h):
+        abspath = os.path.abspath(path)
+
+        if abspath.startswith(model_dir):
+            name = abspath.replace(model_dir, '')
+        else:
+            name = os.path.basename(path)
+
+        if name.startswith("\\") or name.startswith("/"):
+            name = name[1:]
+
+        return f'{name} [{h}]'
+
+    cmd_ckpt = shared.cmd_opts.ckpt
+    if os.path.exists(cmd_ckpt):
+        h = model_hash(cmd_ckpt)
+        title = modeltitle(cmd_ckpt, h)
+        checkpoints_list[title] = CheckpointInfo(cmd_ckpt, title, h)
+    elif cmd_ckpt is not None and cmd_ckpt != shared.default_sd_model_file:
+        print(f"Checkpoint in --ckpt argument not found: {cmd_ckpt}", file=sys.stderr)
+
+    if os.path.exists(model_dir):
+        for filename in glob.glob(model_dir + '/**/*.ckpt', recursive=True):
+            h = model_hash(filename)
+            title = modeltitle(filename, h)
+            checkpoints_list[title] = CheckpointInfo(filename, title, h)
+
+
+def model_hash(filename):
+    try:
+        with open(filename, "rb") as file:
+            import hashlib
+            m = hashlib.sha256()
+
+            file.seek(0x100000)
+            m.update(file.read(0x10000))
+            return m.hexdigest()[0:8]
+    except FileNotFoundError:
+        return 'NOFILE'
+
+
+def select_checkpoint():
+    model_checkpoint = shared.opts.sd_model_checkpoint
+    checkpoint_info = checkpoints_list.get(model_checkpoint, None)
+    if checkpoint_info is not None:
+        return checkpoint_info
+
+    if len(checkpoints_list) == 0:
+        print(f"No checkpoints found. When searching for checkpoints, looked at:", file=sys.stderr)
+        print(f" - file {os.path.abspath(shared.cmd_opts.ckpt)}", file=sys.stderr)
+        print(f" - directory {os.path.abspath(shared.cmd_opts.ckpt_dir)}", file=sys.stderr)
+        print(f"Can't run without a checkpoint. Find and place a .ckpt file into any of those locations. The program will exit.", file=sys.stderr)
+        exit(1)
+
+    checkpoint_info = next(iter(checkpoints_list.values()))
+    if model_checkpoint is not None:
+        print(f"Checkpoint {model_checkpoint} not found; loading fallback {checkpoint_info.title}", file=sys.stderr)
+
+    return checkpoint_info
+
+
+def load_model_weights(model, checkpoint_file, sd_model_hash):
+    print(f"Loading weights [{sd_model_hash}] from {checkpoint_file}")
+
+    pl_sd = torch.load(checkpoint_file, map_location="cpu")
+    if "global_step" in pl_sd:
+        print(f"Global Step: {pl_sd['global_step']}")
+    sd = pl_sd["state_dict"]
+
+    model.load_state_dict(sd, strict=False)
+
+    if shared.cmd_opts.opt_channelslast:
+        model.to(memory_format=torch.channels_last)
+
+    if not shared.cmd_opts.no_half:
+        model.half()
+
+    model.sd_model_hash = sd_model_hash
+    model.sd_model_checkpint = checkpoint_file
+
+
+def load_model():
+    from modules import lowvram, sd_hijack
+    checkpoint_info = select_checkpoint()
+
+    sd_config = OmegaConf.load(shared.cmd_opts.config)
+    sd_model = instantiate_from_config(sd_config.model)
+    load_model_weights(sd_model, checkpoint_info.filename, checkpoint_info.hash)
+
+    if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
+        lowvram.setup_for_low_vram(sd_model, shared.cmd_opts.medvram)
+    else:
+        sd_model.to(shared.device)
+
+    sd_hijack.model_hijack.hijack(sd_model)
+
+    sd_model.eval()
+
+    print(f"Model loaded.")
+    return sd_model
+
+
+def reload_model_weights(sd_model, info=None):
+    from modules import lowvram, devices
+    checkpoint_info = info or select_checkpoint()
+
+    if sd_model.sd_model_checkpint == checkpoint_info.filename:
+        return
+
+    if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
+        lowvram.send_everything_to_cpu()
+    else:
+        sd_model.to(devices.cpu)
+
+    load_model_weights(sd_model, checkpoint_info.filename, checkpoint_info.hash)
+
+    if not shared.cmd_opts.lowvram and not shared.cmd_opts.medvram:
+        sd_model.to(devices.device)
+
+    print(f"Weights loaded.")
+    return sd_model

modules/sd_samplers.py

@@ -7,6 +7,7 @@ from PIL import Image
 import k_diffusion.sampling
 import ldm.models.diffusion.ddim
 import ldm.models.diffusion.plms
+from modules import prompt_parser
 
 from modules.shared import opts, cmd_opts, state
 import modules.shared as shared
@@ -37,6 +38,17 @@ samplers = [
 samplers_for_img2img = [x for x in samplers if x.name != 'PLMS']
 
 
+def setup_img2img_steps(p, steps=None):
+    if opts.img2img_fix_steps or steps is not None:
+        steps = int((steps or p.steps) / min(p.denoising_strength, 0.999)) if p.denoising_strength > 0 else 0
+        t_enc = p.steps - 1
+    else:
+        steps = p.steps
+        t_enc = int(min(p.denoising_strength, 0.999) * steps)
+
+    return steps, t_enc
+
+
 def sample_to_image(samples):
     x_sample = shared.sd_model.decode_first_stage(samples[0:1].type(shared.sd_model.dtype))[0]
     x_sample = torch.clamp((x_sample + 1.0) / 2.0, min=0.0, max=1.0)
@@ -53,20 +65,6 @@ def store_latent(decoded):
             shared.state.current_image = sample_to_image(decoded)
 
 
-def p_sample_ddim_hook(sampler_wrapper, x_dec, cond, ts, *args, **kwargs):
-    if sampler_wrapper.mask is not None:
-        img_orig = sampler_wrapper.sampler.model.q_sample(sampler_wrapper.init_latent, ts)
-        x_dec = img_orig * sampler_wrapper.mask + sampler_wrapper.nmask * x_dec
-
-    res = sampler_wrapper.orig_p_sample_ddim(x_dec, cond, ts, *args, **kwargs)
-
-    if sampler_wrapper.mask is not None:
-        store_latent(sampler_wrapper.init_latent * sampler_wrapper.mask + sampler_wrapper.nmask * res[1])
-    else:
-        store_latent(res[1])
-
-    return res
-
 
 def extended_tdqm(sequence, *args, desc=None, **kwargs):
     state.sampling_steps = len(sequence)
@@ -94,43 +92,66 @@ class VanillaStableDiffusionSampler:
         self.nmask = None
         self.init_latent = None
         self.sampler_noises = None
+        self.step = 0
 
     def number_of_needed_noises(self, p):
         return 0
 
-    def sample_img2img(self, p, x, noise, conditioning, unconditional_conditioning):
-        t_enc = int(min(p.denoising_strength, 0.999) * p.steps)
+    def p_sample_ddim_hook(self, x_dec, cond, ts, unconditional_conditioning, *args, **kwargs):
+        cond = prompt_parser.reconstruct_cond_batch(cond, self.step)
+        unconditional_conditioning = prompt_parser.reconstruct_cond_batch(unconditional_conditioning, self.step)
+
+        if self.mask is not None:
+            img_orig = self.sampler.model.q_sample(self.init_latent, ts)
+            x_dec = img_orig * self.mask + self.nmask * x_dec
+
+        res = self.orig_p_sample_ddim(x_dec, cond, ts, unconditional_conditioning=unconditional_conditioning, *args, **kwargs)
+
+        if self.mask is not None:
+            store_latent(self.init_latent * self.mask + self.nmask * res[1])
+        else:
+            store_latent(res[1])
+
+        self.step += 1
+        return res
+
+    def sample_img2img(self, p, x, noise, conditioning, unconditional_conditioning, steps=None):
+        steps, t_enc = setup_img2img_steps(p, steps)
 
         # existing code fails with cetain step counts, like 9
         try:
-            self.sampler.make_schedule(ddim_num_steps=p.steps, verbose=False)
+            self.sampler.make_schedule(ddim_num_steps=steps, verbose=False)
         except Exception:
-            self.sampler.make_schedule(ddim_num_steps=p.steps+1, verbose=False)
+            self.sampler.make_schedule(ddim_num_steps=steps+1, verbose=False)
 
         x1 = self.sampler.stochastic_encode(x, torch.tensor([t_enc] * int(x.shape[0])).to(shared.device), noise=noise)
 
-        self.sampler.p_sample_ddim = lambda x_dec, cond, ts, *args, **kwargs: p_sample_ddim_hook(self, x_dec, cond, ts, *args, **kwargs)
-        self.mask = p.mask
-        self.nmask = p.nmask
-        self.init_latent = p.init_latent
+        self.sampler.p_sample_ddim = self.p_sample_ddim_hook
+        self.mask = p.mask if hasattr(p, 'mask') else None
+        self.nmask = p.nmask if hasattr(p, 'nmask') else None
+        self.init_latent = x
+        self.step = 0
 
         samples = self.sampler.decode(x1, conditioning, t_enc, unconditional_guidance_scale=p.cfg_scale, unconditional_conditioning=unconditional_conditioning)
 
         return samples
 
-    def sample(self, p, x, conditioning, unconditional_conditioning):
+    def sample(self, p, x, conditioning, unconditional_conditioning, steps=None):
         for fieldname in ['p_sample_ddim', 'p_sample_plms']:
             if hasattr(self.sampler, fieldname):
-                setattr(self.sampler, fieldname, lambda x_dec, cond, ts, *args, **kwargs: p_sample_ddim_hook(self, x_dec, cond, ts, *args, **kwargs))
+                setattr(self.sampler, fieldname, self.p_sample_ddim_hook)
         self.mask = None
         self.nmask = None
         self.init_latent = None
+        self.step = 0
+
+        steps = steps or p.steps
 
         # existing code fails with cetin step counts, like 9
         try:
-            samples_ddim, _ = self.sampler.sample(S=p.steps, conditioning=conditioning, batch_size=int(x.shape[0]), shape=x[0].shape, verbose=False, unconditional_guidance_scale=p.cfg_scale, unconditional_conditioning=unconditional_conditioning, x_T=x)
+            samples_ddim, _ = self.sampler.sample(S=steps, conditioning=conditioning, batch_size=int(x.shape[0]), shape=x[0].shape, verbose=False, unconditional_guidance_scale=p.cfg_scale, unconditional_conditioning=unconditional_conditioning, x_T=x)
         except Exception:
-            samples_ddim, _ = self.sampler.sample(S=p.steps+1, conditioning=conditioning, batch_size=int(x.shape[0]), shape=x[0].shape, verbose=False, unconditional_guidance_scale=p.cfg_scale, unconditional_conditioning=unconditional_conditioning, x_T=x)
+            samples_ddim, _ = self.sampler.sample(S=steps+1, conditioning=conditioning, batch_size=int(x.shape[0]), shape=x[0].shape, verbose=False, unconditional_guidance_scale=p.cfg_scale, unconditional_conditioning=unconditional_conditioning, x_T=x)
 
         return samples_ddim
 
@@ -142,8 +163,12 @@ class CFGDenoiser(torch.nn.Module):
         self.mask = None
         self.nmask = None
         self.init_latent = None
+        self.step = 0
 
     def forward(self, x, sigma, uncond, cond, cond_scale):
+        cond = prompt_parser.reconstruct_cond_batch(cond, self.step)
+        uncond = prompt_parser.reconstruct_cond_batch(uncond, self.step)
+
         if shared.batch_cond_uncond:
             x_in = torch.cat([x] * 2)
             sigma_in = torch.cat([sigma] * 2)
@@ -158,10 +183,12 @@ class CFGDenoiser(torch.nn.Module):
         if self.mask is not None:
             denoised = self.init_latent * self.mask + self.nmask * denoised
 
+        self.step += 1
+
         return denoised
 
 
-def extended_trange(count, *args, **kwargs):
+def extended_trange(sampler, count, *args, **kwargs):
     state.sampling_steps = count
     state.sampling_step = 0
 
@@ -169,6 +196,9 @@ def extended_trange(count, *args, **kwargs):
         if state.interrupted:
             break
 
+        if sampler.stop_at is not None and x > sampler.stop_at:
+            break
+
         yield x
 
         state.sampling_step += 1
@@ -191,12 +221,13 @@ class TorchHijack:
 
 class KDiffusionSampler:
     def __init__(self, funcname, sd_model):
-        self.model_wrap = k_diffusion.external.CompVisDenoiser(sd_model)
+        self.model_wrap = k_diffusion.external.CompVisDenoiser(sd_model, quantize=shared.opts.enable_quantization)
         self.funcname = funcname
         self.func = getattr(k_diffusion.sampling, self.funcname)
         self.model_wrap_cfg = CFGDenoiser(self.model_wrap)
         self.sampler_noises = None
         self.sampler_noise_index = 0
+        self.stop_at = None
 
     def callback_state(self, d):
         store_latent(d["denoised"])
@@ -215,38 +246,47 @@ class KDiffusionSampler:
         self.sampler_noise_index += 1
         return res
 
-    def sample_img2img(self, p, x, noise, conditioning, unconditional_conditioning):
-        t_enc = int(min(p.denoising_strength, 0.999) * p.steps)
-        sigmas = self.model_wrap.get_sigmas(p.steps)
+    def sample_img2img(self, p, x, noise, conditioning, unconditional_conditioning, steps=None):
+        steps, t_enc = setup_img2img_steps(p, steps)
 
-        noise = noise * sigmas[p.steps - t_enc - 1]
+        sigmas = self.model_wrap.get_sigmas(steps)
+
+        noise = noise * sigmas[steps - t_enc - 1]
 
         xi = x + noise
 
-        sigma_sched = sigmas[p.steps - t_enc - 1:]
+        sigma_sched = sigmas[steps - t_enc - 1:]
 
-        self.model_wrap_cfg.mask = p.mask
-        self.model_wrap_cfg.nmask = p.nmask
-        self.model_wrap_cfg.init_latent = p.init_latent
+        self.model_wrap_cfg.mask = p.mask if hasattr(p, 'mask') else None
+        self.model_wrap_cfg.nmask = p.nmask if hasattr(p, 'nmask') else None
+        self.model_wrap_cfg.init_latent = x
+        self.model_wrap.step = 0
+        self.sampler_noise_index = 0
 
         if hasattr(k_diffusion.sampling, 'trange'):
-            k_diffusion.sampling.trange = lambda *args, **kwargs: extended_trange(*args, **kwargs)
+            k_diffusion.sampling.trange = lambda *args, **kwargs: extended_trange(self, *args, **kwargs)
 
         if self.sampler_noises is not None:
             k_diffusion.sampling.torch = TorchHijack(self)
 
         return self.func(self.model_wrap_cfg, xi, sigma_sched, extra_args={'cond': conditioning, 'uncond': unconditional_conditioning, 'cond_scale': p.cfg_scale}, disable=False, callback=self.callback_state)
 
-    def sample(self, p, x, conditioning, unconditional_conditioning):
-        sigmas = self.model_wrap.get_sigmas(p.steps)
+    def sample(self, p, x, conditioning, unconditional_conditioning, steps=None):
+        steps = steps or p.steps
+
+        sigmas = self.model_wrap.get_sigmas(steps)
         x = x * sigmas[0]
 
+        self.model_wrap_cfg.step = 0
+        self.sampler_noise_index = 0
+
         if hasattr(k_diffusion.sampling, 'trange'):
-            k_diffusion.sampling.trange = lambda *args, **kwargs: extended_trange(*args, **kwargs)
+            k_diffusion.sampling.trange = lambda *args, **kwargs: extended_trange(self, *args, **kwargs)
 
         if self.sampler_noises is not None:
             k_diffusion.sampling.torch = TorchHijack(self)
 
-        samples_ddim = self.func(self.model_wrap_cfg, x, sigmas, extra_args={'cond': conditioning, 'uncond': unconditional_conditioning, 'cond_scale': p.cfg_scale}, disable=False, callback=self.callback_state)
-        return samples_ddim
+        samples = self.func(self.model_wrap_cfg, x, sigmas, extra_args={'cond': conditioning, 'uncond': unconditional_conditioning, 'cond_scale': p.cfg_scale}, disable=False, callback=self.callback_state)
+
+        return samples
 

modules/shared.py

@@ -4,7 +4,6 @@ import json
 import os
 
 import gradio as gr
-import torch
 import tqdm
 
 import modules.artists
@@ -12,30 +11,33 @@ from modules.paths import script_path, sd_path
 from modules.devices import get_optimal_device
 import modules.styles
 import modules.interrogate
+import modules.memmon
+import modules.sd_models
 
 sd_model_file = os.path.join(script_path, 'model.ckpt')
-if not os.path.exists(sd_model_file):
-    sd_model_file = "models/ldm/stable-diffusion-v1/model.ckpt"
+default_sd_model_file = sd_model_file
 
 parser = argparse.ArgumentParser()
 parser.add_argument("--config", type=str, default=os.path.join(sd_path, "configs/stable-diffusion/v1-inference.yaml"), help="path to config which constructs model",)
-parser.add_argument("--ckpt", type=str, default=os.path.join(sd_path, sd_model_file), help="path to checkpoint of model",)
+parser.add_argument("--ckpt", type=str, default=sd_model_file, help="path to checkpoint of stable diffusion model; this checkpoint will be added to the list of checkpoints and loaded by default if you don't have a checkpoint selected in settings",)
+parser.add_argument("--ckpt-dir", type=str, default=os.path.join(script_path, 'models'), help="path to directory with stable diffusion checkpoints",)
 parser.add_argument("--gfpgan-dir", type=str, help="GFPGAN directory", default=('./src/gfpgan' if os.path.exists('./src/gfpgan') else './GFPGAN'))
 parser.add_argument("--gfpgan-model", type=str, help="GFPGAN model file name", default='GFPGANv1.3.pth')
 parser.add_argument("--no-half", action='store_true', help="do not switch the model to 16-bit floats")
-parser.add_argument("--no-progressbar-hiding", action='store_true', help="do not hide progressbar in gradio UI (we hide it because it slows down ML if you have hardware accleration in browser)")
+parser.add_argument("--no-progressbar-hiding", action='store_true', help="do not hide progressbar in gradio UI (we hide it because it slows down ML if you have hardware acceleration in browser)")
 parser.add_argument("--max-batch-count", type=int, default=16, help="maximum batch count value for the UI")
 parser.add_argument("--embeddings-dir", type=str, default=os.path.join(script_path, 'embeddings'), help="embeddings directory for textual inversion (default: embeddings)")
 parser.add_argument("--allow-code", action='store_true', help="allow custom script execution from webui")
 parser.add_argument("--medvram", action='store_true', help="enable stable diffusion model optimizations for sacrificing a little speed for low VRM usage")
 parser.add_argument("--lowvram", action='store_true', help="enable stable diffusion model optimizations for sacrificing a lot of speed for very low VRM usage")
-parser.add_argument("--always-batch-cond-uncond", action='store_true', help="a workaround test; may help with speed if you use --lowvram")
+parser.add_argument("--always-batch-cond-uncond", action='store_true', help="disables cond/uncond batching that is enabled to save memory with --medvram or --lowvram")
 parser.add_argument("--unload-gfpgan", action='store_true', help="does not do anything.")
 parser.add_argument("--precision", type=str, help="evaluate at this precision", choices=["full", "autocast"], default="autocast")
 parser.add_argument("--share", action='store_true', help="use share=True for gradio and make the UI accessible through their site (doesn't work for me but you might have better luck)")
 parser.add_argument("--esrgan-models-path", type=str, help="path to directory with ESRGAN models", default=os.path.join(script_path, 'ESRGAN'))
-parser.add_argument("--opt-split-attention", action='store_true', help="enable optimization that reduce vram usage by a lot for about 10%% decrease in performance")
-parser.add_argument("--opt-split-attention-v1", action='store_true', help="enable older version of --opt-split-attention optimization")
+parser.add_argument("--opt-split-attention", action='store_true', help="does not do anything")
+parser.add_argument("--disable-opt-split-attention", action='store_true', help="disable an optimization that reduces vram usage by a lot")
+parser.add_argument("--opt-split-attention-v1", action='store_true', help="enable older version of split attention optimization that does not consume all the VRAM it can find")
 parser.add_argument("--listen", action='store_true', help="launch gradio with 0.0.0.0 as server name, allowing to respond to network requests")
 parser.add_argument("--port", type=int, help="launch gradio with given server port, you need root/admin rights for ports < 1024, defaults to 7860 if available", default=None)
 parser.add_argument("--show-negative-prompt", action='store_true', help="does not do anything", default=False)
@@ -45,9 +47,13 @@ parser.add_argument("--ui-settings-file", type=str, help="filename to use for ui
 parser.add_argument("--gradio-debug",  action='store_true', help="launch gradio with --debug option")
 parser.add_argument("--gradio-auth", type=str, help='set gradio authentication like "username:password"; or comma-delimit multiple like "u1:p1,u2:p2,u3:p3"', default=None)
 parser.add_argument("--opt-channelslast", action='store_true', help="change memory type for stable diffusion to channels last")
-
+parser.add_argument("--styles-file", type=str, help="filename to use for styles", default=os.path.join(script_path, 'styles.csv'))
+parser.add_argument("--autolaunch", action='store_true', help="open the webui URL in the system's default browser upon launch", default=False)
 cmd_opts = parser.parse_args()
 
+if cmd_opts.opt_split_attention:
+    print("Information: --opt-split-attention is now the default. To remove this message, remove --opt-split-attention from command line arguments. To disable the optimization, use --disable-opt-split-attention")
+
 device = get_optimal_device()
 
 batch_cond_uncond = cmd_opts.always_batch_cond_uncond or not (cmd_opts.lowvram or cmd_opts.medvram)
@@ -79,20 +85,24 @@ state = State()
 
 artist_db = modules.artists.ArtistsDatabase(os.path.join(script_path, 'artists.csv'))
 
-styles_filename = os.path.join(script_path, 'styles.csv')
-prompt_styles = modules.styles.load_styles(styles_filename)
+styles_filename = cmd_opts.styles_file
+prompt_styles = modules.styles.StyleDatabase(styles_filename)
 
 interrogator = modules.interrogate.InterrogateModels("interrogate")
 
 face_restorers = []
 
+modules.sd_models.list_models()
+
+
 class Options:
     class OptionInfo:
-        def __init__(self, default=None, label="", component=None, component_args=None):
+        def __init__(self, default=None, label="", component=None, component_args=None, onchange=None):
             self.default = default
             self.label = label
             self.component = component
             self.component_args = component_args
+            self.onchange = onchange
 
     data = None
     hide_dirs = {"visible": False} if cmd_opts.hide_ui_dir_config else None
@@ -109,10 +119,11 @@ class Options:
         "outdir_txt2img_grids": OptionInfo("outputs/txt2img-grids", 'Output directory for txt2img grids', component_args=hide_dirs),
         "outdir_img2img_grids": OptionInfo("outputs/img2img-grids", 'Output directory for img2img grids', component_args=hide_dirs),
         "outdir_save": OptionInfo("log/images", "Directory for saving images using the Save button", component_args=hide_dirs),
-        "samples_save": OptionInfo(True, "Save indiviual samples"),
+        "samples_save": OptionInfo(True, "Always save all generated images"),
+        "save_selected_only": OptionInfo(False, "When using 'Save' button, only save a single selected image"),
         "samples_format": OptionInfo('png', 'File format for individual samples'),
         "filter_nsfw": OptionInfo(False, "Filter NSFW content"),
-        "grid_save": OptionInfo(True, "Save image grids"),
+        "grid_save": OptionInfo(True, "Always save all generated image grids"),
         "return_grid": OptionInfo(True, "Show grid in results for web"),
         "grid_format": OptionInfo('png', 'File format for grids'),
         "grid_extended_filename": OptionInfo(False, "Add extended info (seed, prompt) to filename when saving grid"),
@@ -123,6 +134,8 @@ class Options:
         "enable_pnginfo": OptionInfo(True, "Save text information about generation parameters as chunks to png files"),
         "add_model_hash_to_info": OptionInfo(False, "Add model hash to generation information"),
         "img2img_color_correction": OptionInfo(False, "Apply color correction to img2img results to match original colors."),
+        "img2img_fix_steps": OptionInfo(False, "With img2img, do exactly the amount of steps the slider specifies (normally you'd do less with less denoising)."),
+        "enable_quantization": OptionInfo(False, "Enable quantization in K samplers for sharper and cleaner results. This may change existing seeds. Requires restart to apply."),
         "font": OptionInfo("", "Font for image grids that have text"),
         "enable_emphasis": OptionInfo(True, "Use (text) to make model pay more attention to text and [text] to make it pay less attention"),
         "enable_batch_seeds": OptionInfo(True, "Make K-diffusion samplers produce same images in a batch as when making a single image"),
@@ -134,6 +147,7 @@ class Options:
         "show_progressbar": OptionInfo(True, "Show progressbar"),
         "show_progress_every_n_steps": OptionInfo(0, "Show show image creation progress every N sampling steps. Set 0 to disable.", gr.Slider, {"minimum": 0, "maximum": 32, "step": 1}),
         "multiple_tqdm": OptionInfo(True, "Add a second progress bar to the console that shows progress for an entire job. Broken in PyCharm console."),
+        "memmon_poll_rate": OptionInfo(8, "VRAM usage polls per second during generation. Set to 0 to disable.", gr.Slider, {"minimum": 0, "maximum": 40, "step":1}),
         "face_restoration_model": OptionInfo(None, "Face restoration model", gr.Radio, lambda: {"choices": [x.name() for x in face_restorers]}),
         "code_former_weight": OptionInfo(0.5, "CodeFormer weight parameter; 0 = maximum effect; 1 = minimum effect", gr.Slider, {"minimum": 0, "maximum": 1, "step": 0.01}),
         "save_images_before_face_restoration": OptionInfo(False, "Save a copy of image before doing face restoration."),
@@ -141,9 +155,11 @@ class Options:
         "interrogate_keep_models_in_memory": OptionInfo(False, "Interrogate: keep models in VRAM"),
         "interrogate_use_builtin_artists": OptionInfo(True, "Interrogate: use artists from artists.csv"),
         "interrogate_clip_num_beams": OptionInfo(1, "Interrogate: num_beams for BLIP", gr.Slider, {"minimum": 1, "maximum": 16, "step": 1}),
-        "interrogate_clip_min_length": OptionInfo(24, "Interrogate: minimum descripton length (excluding artists, etc..)", gr.Slider, {"minimum": 1, "maximum": 128, "step": 1}),
-        "interrogate_clip_max_length": OptionInfo(48, "Interrogate: maximum descripton length", gr.Slider, {"minimum": 1, "maximum": 256, "step": 1}),
+        "interrogate_clip_min_length": OptionInfo(24, "Interrogate: minimum description length (excluding artists, etc..)", gr.Slider, {"minimum": 1, "maximum": 128, "step": 1}),
+        "interrogate_clip_max_length": OptionInfo(48, "Interrogate: maximum description length", gr.Slider, {"minimum": 1, "maximum": 256, "step": 1}),
         "interrogate_clip_dict_limit": OptionInfo(1500, "Interrogate: maximum number of lines in text file (0 = No limit)"),
+        "sd_model_checkpoint": OptionInfo(None, "Stable Diffusion checkpoint", gr.Radio, lambda: {"choices": [x.title for x in modules.sd_models.checkpoints_list.values()]}),
+        "js_modal_lightbox": OptionInfo(True, "Enable full page image viewer"),
     }
 
     def __init__(self):
@@ -174,6 +190,14 @@ class Options:
         with open(filename, "r", encoding="utf8") as file:
             self.data = json.load(file)
 
+    def onchange(self, key, func):
+        item = self.data_labels.get(key)
+        item.onchange = func
+
+    def dumpjson(self):
+        d = {k: self.data.get(k, self.data_labels.get(k).default) for k in self.data_labels.keys()}
+        return json.dumps(d)
+
 
 opts = Options()
 if os.path.exists(config_filename):
@@ -182,7 +206,6 @@ if os.path.exists(config_filename):
 sd_upscalers = []
 
 sd_model = None
-sd_model_hash = ''
 
 progress_print_out = sys.stdout
 
@@ -213,3 +236,6 @@ class TotalTQDM:
 
 
 total_tqdm = TotalTQDM()
+
+mem_mon = modules.memmon.MemUsageMonitor("MemMon", device, opts)
+mem_mon.start()

modules/styles.py

@@ -20,49 +20,67 @@ class PromptStyle(typing.NamedTuple):
     negative_prompt: str
 
 
-def load_styles(path: str) -> dict[str, PromptStyle]:
-    styles = {"None": PromptStyle("None", "", "")}
+def merge_prompts(style_prompt: str, prompt: str) -> str:
+    if "{prompt}" in style_prompt:
+        res = style_prompt.replace("{prompt}", prompt)
+    else:
+        parts = filter(None, (prompt.strip(), style_prompt.strip()))
+        res = ", ".join(parts)
+
+    return res
+
+
+def apply_styles_to_prompt(prompt, styles):
+    for style in styles:
+        prompt = merge_prompts(style, prompt)
+
+    return prompt
+
+
+class StyleDatabase:
+    def __init__(self, path: str):
+        self.no_style = PromptStyle("None", "", "")
+        self.styles = {"None": self.no_style}
+
+        if not os.path.exists(path):
+            return
 
-    if os.path.exists(path):
         with open(path, "r", encoding="utf8", newline='') as file:
             reader = csv.DictReader(file)
             for row in reader:
                 # Support loading old CSV format with "name, text"-columns
                 prompt = row["prompt"] if "prompt" in row else row["text"]
                 negative_prompt = row.get("negative_prompt", "")
-                styles[row["name"]] = PromptStyle(row["name"], prompt, negative_prompt)
+                self.styles[row["name"]] = PromptStyle(row["name"], prompt, negative_prompt)
 
-    return styles
+    def apply_styles_to_prompt(self, prompt, styles):
+        return apply_styles_to_prompt(prompt, [self.styles.get(x, self.no_style).prompt for x in styles])
 
+    def apply_negative_styles_to_prompt(self, prompt, styles):
+        return apply_styles_to_prompt(prompt, [self.styles.get(x, self.no_style).negative_prompt for x in styles])
 
-def merge_prompts(style_prompt: str, prompt: str) -> str:
-    parts = filter(None, (prompt.strip(), style_prompt.strip()))
-    return ", ".join(parts)
+    def apply_styles(self, p: StableDiffusionProcessing) -> None:
+        if isinstance(p.prompt, list):
+            p.prompt = [self.apply_styles_to_prompt(prompt, p.styles) for prompt in p.prompt]
+        else:
+            p.prompt = self.apply_styles_to_prompt(p.prompt, p.styles)
 
+        if isinstance(p.negative_prompt, list):
+            p.negative_prompt = [self.apply_negative_styles_to_prompt(prompt, p.styles) for prompt in p.negative_prompt]
+        else:
+            p.negative_prompt = self.apply_negative_styles_to_prompt(p.negative_prompt, p.styles)
 
-def apply_style(processing: StableDiffusionProcessing, style: PromptStyle) -> None:
-    if isinstance(processing.prompt, list):
-        processing.prompt = [merge_prompts(style.prompt, p) for p in processing.prompt]
-    else:
-        processing.prompt = merge_prompts(style.prompt, processing.prompt)
+    def save_styles(self, path: str) -> None:
+        # Write to temporary file first, so we don't nuke the file if something goes wrong
+        fd, temp_path = tempfile.mkstemp(".csv")
+        with os.fdopen(fd, "w", encoding="utf8", newline='') as file:
+            # _fields is actually part of the public API: typing.NamedTuple is a replacement for collections.NamedTuple,
+            # and collections.NamedTuple has explicit documentation for accessing _fields. Same goes for _asdict()
+            writer = csv.DictWriter(file, fieldnames=PromptStyle._fields)
+            writer.writeheader()
+            writer.writerows(style._asdict() for k,     style in self.styles.items())
 
-    if isinstance(processing.negative_prompt, list):
-        processing.negative_prompt = [merge_prompts(style.negative_prompt, p) for p in processing.negative_prompt]
-    else:
-        processing.negative_prompt = merge_prompts(style.negative_prompt, processing.negative_prompt)
-
-
-def save_styles(path: str, styles: abc.Iterable[PromptStyle]) -> None:
-    # Write to temporary file first, so we don't nuke the file if something goes wrong
-    fd, temp_path = tempfile.mkstemp(".csv")
-    with os.fdopen(fd, "w", encoding="utf8", newline='') as file:
-        # _fields is actually part of the public API: typing.NamedTuple is a replacement for collections.NamedTuple,
-        # and collections.NamedTuple has explicit documentation for accessing _fields. Same goes for _asdict()
-        writer = csv.DictWriter(file, fieldnames=PromptStyle._fields)
-        writer.writeheader()
-        writer.writerows(style._asdict() for style in styles)
-
-    # Always keep a backup file around
-    if os.path.exists(path):
-        shutil.move(path, path + ".bak")
-    shutil.move(temp_path, path)
+        # Always keep a backup file around
+        if os.path.exists(path):
+            shutil.move(path, path + ".bak")
+        shutil.move(temp_path, path)

modules/txt2img.py

@@ -6,13 +6,13 @@ import modules.processing as processing
 from modules.ui import plaintext_to_html
 
 
-def txt2img(prompt: str, negative_prompt: str, prompt_style: str, steps: int, sampler_index: int, restore_faces: bool, tiling: bool, n_iter: int, batch_size: int, cfg_scale: float, seed: int, subseed: int, subseed_strength: float, seed_resize_from_h: int, seed_resize_from_w: int, height: int, width: int, *args):
+def txt2img(prompt: str, negative_prompt: str, prompt_style: str, prompt_style2: str, steps: int, sampler_index: int, restore_faces: bool, tiling: bool, n_iter: int, batch_size: int, cfg_scale: float, seed: int, subseed: int, subseed_strength: float, seed_resize_from_h: int, seed_resize_from_w: int, height: int, width: int, enable_hr: bool, scale_latent: bool, denoising_strength: float, *args):
     p = StableDiffusionProcessingTxt2Img(
         sd_model=shared.sd_model,
         outpath_samples=opts.outdir_samples or opts.outdir_txt2img_samples,
         outpath_grids=opts.outdir_grids or opts.outdir_txt2img_grids,
         prompt=prompt,
-        prompt_style=prompt_style,
+        styles=[prompt_style, prompt_style2],
         negative_prompt=negative_prompt,
         seed=seed,
         subseed=subseed,
@@ -28,6 +28,9 @@ def txt2img(prompt: str, negative_prompt: str, prompt_style: str, steps: int, sa
         height=height,
         restore_faces=restore_faces,
         tiling=tiling,
+        enable_hr=enable_hr,
+        scale_latent=scale_latent,
+        denoising_strength=denoising_strength,
     )
 
     print(f"\ntxt2img: {prompt}", file=shared.progress_print_out)

modules/ui.py

@@ -53,6 +53,12 @@ css_hide_progressbar = """
 .meta-text { display:none!important; }
 """
 
+# Using constants for these since the variation selector isn't visible.
+# Important that they exactly match script.js for tooltip to work.
+random_symbol = '\U0001f3b2\ufe0f'  # 🎲️
+reuse_symbol = '\u267b\ufe0f'  # ♻️
+
+
 def plaintext_to_html(text):
     text = "<p>" + "<br>\n".join([f"{html.escape(x)}" for x in text.split('\n')]) + "</p>"
     return text
@@ -80,7 +86,7 @@ def send_gradio_gallery_to_image(x):
     return image_from_url_text(x[0])
 
 
-def save_files(js_data, images):
+def save_files(js_data, images, index):
     import csv
 
     os.makedirs(opts.outdir_save, exist_ok=True)
@@ -88,6 +94,10 @@ def save_files(js_data, images):
     filenames = []
 
     data = json.loads(js_data)
+    
+    if index > -1 and opts.save_selected_only and (index > 0 or not opts.return_grid): # ensures we are looking at a specific non-grid picture, and we have save_selected_only
+        images = [images[index]]
+        data["seed"] += (index - 1 if opts.return_grid else index)
 
     with open(os.path.join(opts.outdir_save, "log.csv"), "a", encoding="utf8", newline='') as file:
         at_start = file.tell() == 0
@@ -115,6 +125,9 @@ def save_files(js_data, images):
 
 def wrap_gradio_call(func):
     def f(*args, **kwargs):
+        run_memmon = opts.memmon_poll_rate > 0 and not shared.mem_mon.disabled
+        if run_memmon:
+            shared.mem_mon.monitor()
         t = time.perf_counter()
 
         try:
@@ -131,8 +144,20 @@ def wrap_gradio_call(func):
 
         elapsed = time.perf_counter() - t
 
+        if run_memmon:
+            mem_stats = {k: -(v//-(1024*1024)) for k, v in shared.mem_mon.stop().items()}
+            active_peak = mem_stats['active_peak']
+            reserved_peak = mem_stats['reserved_peak']
+            sys_peak = mem_stats['system_peak']
+            sys_total = mem_stats['total']
+            sys_pct = round(sys_peak/max(sys_total, 1) * 100, 2)
+
+            vram_html = f"<p class='vram'>Torch active/reserved: {active_peak}/{reserved_peak} MiB, <wbr>Sys VRAM: {sys_peak}/{sys_total} MiB ({sys_pct}%)</p>"
+        else:
+            vram_html = ''
+
         # last item is always HTML
-        res[-1] = res[-1] + f"<p class='performance'>Time taken: {elapsed:.2f}s</p>"
+        res[-1] += f"<div class='performance'><p class='time'>Time taken: <wbr>{elapsed:.2f}s</p>{vram_html}</div>"
 
         shared.state.interrupted = False
 
@@ -142,7 +167,6 @@ def wrap_gradio_call(func):
 
 
 def check_progress_call():
-
     if shared.state.job_count == 0:
         return "", gr_show(False), gr_show(False)
 
@@ -179,6 +203,14 @@ def check_progress_call():
     return f"<span style='display: none'>{time.time()}</span><p>{progressbar}</p>", preview_visibility, image
 
 
+def check_progress_call_initial():
+    shared.state.job_count = -1
+    shared.state.current_latent = None
+    shared.state.current_image = None
+
+    return check_progress_call()
+
+
 def roll_artist(prompt):
     allowed_cats = set([x for x in shared.artist_db.categories() if len(opts.random_artist_categories)==0 or x in opts.random_artist_categories])
     artist = random.choice([x for x in shared.artist_db.artists if x.category in allowed_cats])
@@ -194,52 +226,25 @@ def visit(x, func, path=""):
         func(path + "/" + str(x.label), x)
 
 
-def create_seed_inputs():
-    with gr.Row():
-        seed = gr.Number(label='Seed', value=-1)
-        subseed = gr.Number(label='Variation seed', value=-1, visible=False)
-        seed_checkbox = gr.Checkbox(label="Extra", elem_id="subseed_show", value=False)
-
-    with gr.Row():
-        subseed_strength = gr.Slider(label='Variation strength', value=0.0, minimum=0, maximum=1, step=0.01, visible=False)
-        seed_resize_from_w = gr.Slider(minimum=0, maximum=2048, step=64, label="Resize seed from width", value=0, visible=False)
-        seed_resize_from_h = gr.Slider(minimum=0, maximum=2048, step=64, label="Resize seed from height", value=0, visible=False)
-
-    def change_visiblity(show):
-
-        return {
-            subseed: gr_show(show),
-            subseed_strength: gr_show(show),
-            seed_resize_from_h: gr_show(show),
-            seed_resize_from_w: gr_show(show),
-        }
-
-    seed_checkbox.change(
-        change_visiblity,
-        inputs=[seed_checkbox],
-        outputs=[
-            subseed,
-            subseed_strength,
-            seed_resize_from_h,
-            seed_resize_from_w
-        ]
-    )
-
-    return seed, subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w
-
-
 def add_style(name: str, prompt: str, negative_prompt: str):
     if name is None:
         return [gr_show(), gr_show()]
 
     style = modules.styles.PromptStyle(name, prompt, negative_prompt)
-    shared.prompt_styles[style.name] = style
+    shared.prompt_styles.styles[style.name] = style
     # Save all loaded prompt styles: this allows us to update the storage format in the future more easily, because we
     # reserialize all styles every time we save them
-    modules.styles.save_styles(shared.styles_filename, shared.prompt_styles.values())
+    shared.prompt_styles.save_styles(shared.styles_filename)
 
-    update = {"visible": True, "choices": list(shared.prompt_styles), "__type__": "update"}
-    return [update, update]
+    update = {"visible": True, "choices": list(shared.prompt_styles.styles), "__type__": "update"}
+    return [update, update, update, update]
+
+
+def apply_styles(prompt, prompt_neg, style1_name, style2_name):
+    prompt = shared.prompt_styles.apply_styles_to_prompt(prompt, [style1_name, style2_name])
+    prompt_neg = shared.prompt_styles.apply_negative_styles_to_prompt(prompt_neg, [style1_name, style2_name])
+
+    return [gr.Textbox.update(value=prompt), gr.Textbox.update(value=prompt_neg), gr.Dropdown.update(value="None"), gr.Dropdown.update(value="None")]
 
 
 def interrogate(image):
@@ -247,15 +252,138 @@ def interrogate(image):
 
     return gr_show(True) if prompt is None else prompt
 
+
+def create_seed_inputs():
+    with gr.Row():
+        with gr.Box():
+            with gr.Row(elem_id='seed_row'):
+                seed = gr.Number(label='Seed', value=-1)
+                seed.style(container=False)
+                random_seed = gr.Button(random_symbol, elem_id='random_seed')
+                reuse_seed = gr.Button(reuse_symbol, elem_id='reuse_seed')
+
+        with gr.Box(elem_id='subseed_show_box'):
+            seed_checkbox = gr.Checkbox(label='Extra', elem_id='subseed_show', value=False)
+
+    # Components to show/hide based on the 'Extra' checkbox
+    seed_extras = []
+
+    with gr.Row(visible=False) as seed_extra_row_1:
+        seed_extras.append(seed_extra_row_1)
+        with gr.Box():
+            with gr.Row(elem_id='subseed_row'):
+                subseed = gr.Number(label='Variation seed', value=-1)
+                subseed.style(container=False)
+                random_subseed = gr.Button(random_symbol, elem_id='random_subseed')
+                reuse_subseed = gr.Button(reuse_symbol, elem_id='reuse_subseed')
+        subseed_strength = gr.Slider(label='Variation strength', value=0.0, minimum=0, maximum=1, step=0.01)
+
+    with gr.Row(visible=False) as seed_extra_row_2:
+        seed_extras.append(seed_extra_row_2)
+        seed_resize_from_w = gr.Slider(minimum=0, maximum=2048, step=64, label="Resize seed from width", value=0)
+        seed_resize_from_h = gr.Slider(minimum=0, maximum=2048, step=64, label="Resize seed from height", value=0)
+
+    random_seed.click(fn=lambda: -1, show_progress=False, inputs=[], outputs=[seed])
+    random_subseed.click(fn=lambda: -1, show_progress=False, inputs=[], outputs=[subseed])
+
+    def change_visibility(show):
+        return {comp: gr_show(show) for comp in seed_extras}
+
+    seed_checkbox.change(change_visibility, show_progress=False, inputs=[seed_checkbox], outputs=seed_extras)
+
+    return seed, reuse_seed, subseed, reuse_subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w
+
+
+def connect_reuse_seed(seed: gr.Number, reuse_seed: gr.Button, generation_info: gr.Textbox, dummy_component, is_subseed):
+    """ Connects a 'reuse (sub)seed' button's click event so that it copies last used
+        (sub)seed value from generation info the to the seed field. If copying subseed and subseed strength
+        was 0, i.e. no variation seed was used, it copies the normal seed value instead."""
+    def copy_seed(gen_info_string: str, index):
+        res = -1
+
+        try:
+            gen_info = json.loads(gen_info_string)
+            index -= gen_info.get('index_of_first_image', 0)
+
+            if is_subseed and gen_info.get('subseed_strength', 0) > 0:
+                all_subseeds = gen_info.get('all_subseeds', [-1])
+                res = all_subseeds[index if 0 <= index < len(all_subseeds) else 0]
+            else:
+                all_seeds = gen_info.get('all_seeds', [-1])
+                res = all_seeds[index if 0 <= index < len(all_seeds) else 0]
+
+        except json.decoder.JSONDecodeError as e:
+            if gen_info_string != '':
+                print("Error parsing JSON generation info:", file=sys.stderr)
+                print(gen_info_string, file=sys.stderr)
+
+        return [res, gr_show(False)]
+
+    reuse_seed.click(
+        fn=copy_seed,
+        _js="(x, y) => [x, selected_gallery_index()]",
+        show_progress=False,
+        inputs=[generation_info, dummy_component],
+        outputs=[seed, dummy_component]
+    )
+
+
+def create_toprow(is_img2img):
+    with gr.Row(elem_id="toprow"):
+        with gr.Column(scale=4):
+            with gr.Row():
+                with gr.Column(scale=8):
+                    with gr.Row():
+                        prompt = gr.Textbox(label="Prompt", elem_id="prompt", show_label=False, placeholder="Prompt", lines=2)
+                        roll = gr.Button('Roll', elem_id="roll", visible=len(shared.artist_db.artists) > 0)
+
+                with gr.Column(scale=1, elem_id="style_pos_col"):
+                    prompt_style = gr.Dropdown(label="Style 1", elem_id="style_index", choices=[k for k, v in shared.prompt_styles.styles.items()], value=next(iter(shared.prompt_styles.styles.keys())), visible=len(shared.prompt_styles.styles) > 1)
+
+            with gr.Row():
+                with gr.Column(scale=8):
+                    negative_prompt = gr.Textbox(label="Negative prompt", elem_id="negative_prompt", show_label=False, placeholder="Negative prompt", lines=2)
+
+                with gr.Column(scale=1, elem_id="style_neg_col"):
+                    prompt_style2 = gr.Dropdown(label="Style 2", elem_id="style2_index", choices=[k for k, v in shared.prompt_styles.styles.items()], value=next(iter(shared.prompt_styles.styles.keys())), visible=len(shared.prompt_styles.styles) > 1)
+
+        with gr.Column(scale=1):
+            with gr.Row():
+                submit = gr.Button('Generate', elem_id="generate", variant='primary')
+
+            with gr.Row():
+                if is_img2img:
+                    interrogate = gr.Button('Interrogate', elem_id="interrogate")
+                else:
+                    interrogate = None
+                prompt_style_apply = gr.Button('Apply style', elem_id="style_apply")
+                save_style = gr.Button('Create style', elem_id="style_create")
+
+    return prompt, roll, prompt_style, negative_prompt, prompt_style2, submit, interrogate, prompt_style_apply, save_style
+
+
+def setup_progressbar(progressbar, preview):
+    check_progress = gr.Button('Check progress', elem_id="check_progress", visible=False)
+    check_progress.click(
+        fn=check_progress_call,
+        show_progress=False,
+        inputs=[],
+        outputs=[progressbar, preview, preview],
+    )
+
+    check_progress_initial = gr.Button('Check progress (first)', elem_id="check_progress_initial", visible=False)
+    check_progress_initial.click(
+        fn=check_progress_call_initial,
+        show_progress=False,
+        inputs=[],
+        outputs=[progressbar, preview, preview],
+    )
+
+
 def create_ui(txt2img, img2img, run_extras, run_pnginfo):
     with gr.Blocks(analytics_enabled=False) as txt2img_interface:
-        with gr.Row(elem_id="toprow"):
-            txt2img_prompt = gr.Textbox(label="Prompt", elem_id="txt2img_prompt", show_label=False, placeholder="Prompt", lines=1)
-            txt2img_negative_prompt = gr.Textbox(label="Negative prompt", elem_id="txt2img_negative_prompt", show_label=False, placeholder="Negative prompt", lines=1)
-            txt2img_prompt_style = gr.Dropdown(label="Style", show_label=False, elem_id="style_index", choices=[k for k, v in shared.prompt_styles.items()], value=next(iter(shared.prompt_styles.keys())), visible=len(shared.prompt_styles) > 1)
-            roll = gr.Button('Roll', elem_id="txt2img_roll", visible=len(shared.artist_db.artists) > 0)
-            submit = gr.Button('Generate', elem_id="txt2img_generate", variant='primary')
-            check_progress = gr.Button('Check progress', elem_id="check_progress", visible=False)
+        txt2img_prompt, roll, txt2img_prompt_style, txt2img_negative_prompt, txt2img_prompt_style2, submit, _, txt2img_prompt_style_apply, txt2img_save_style = create_toprow(is_img2img=False)
+        dummy_component = gr.Label(visible=False)
 
         with gr.Row().style(equal_height=False):
             with gr.Column(variant='panel'):
@@ -265,6 +393,11 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 with gr.Row():
                     restore_faces = gr.Checkbox(label='Restore faces', value=False, visible=len(shared.face_restorers) > 1)
                     tiling = gr.Checkbox(label='Tiling', value=False)
+                    enable_hr = gr.Checkbox(label='Highres. fix', value=False)
+
+                with gr.Row(visible=False) as hr_options:
+                    scale_latent = gr.Checkbox(label='Scale latent', value=True)
+                    denoising_strength = gr.Slider(minimum=0.0, maximum=1.0, step=0.01, label='Denoising strength', value=0.7)
 
                 with gr.Row():
                     batch_count = gr.Slider(minimum=1, maximum=cmd_opts.max_batch_count, step=1, label='Batch count', value=1)
@@ -276,16 +409,19 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     width = gr.Slider(minimum=64, maximum=2048, step=64, label="Width", value=512)
                     height = gr.Slider(minimum=64, maximum=2048, step=64, label="Height", value=512)
 
-                seed, subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w = create_seed_inputs()
+                seed, reuse_seed, subseed, reuse_subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w = create_seed_inputs()
 
                 with gr.Group():
                     custom_inputs = modules.scripts.scripts_txt2img.setup_ui(is_img2img=False)
 
             with gr.Column(variant='panel'):
+                progressbar = gr.HTML(elem_id="progressbar")
+
                 with gr.Group():
                     txt2img_preview = gr.Image(elem_id='txt2img_preview', visible=False)
                     txt2img_gallery = gr.Gallery(label='Output', elem_id='txt2img_gallery').style(grid=4)
 
+                setup_progressbar(progressbar, txt2img_preview)
 
                 with gr.Group():
                     with gr.Row():
@@ -294,14 +430,13 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                         send_to_inpaint = gr.Button('Send to inpaint')
                         send_to_extras = gr.Button('Send to extras')
                         interrupt = gr.Button('Interrupt')
-                        txt2img_save_style = gr.Button('Save prompt as style')
-
-                progressbar = gr.HTML(elem_id="progressbar")
 
                 with gr.Group():
                     html_info = gr.HTML()
                     generation_info = gr.Textbox(visible=False)
 
+            connect_reuse_seed(seed, reuse_seed, generation_info, dummy_component, is_subseed=False)
+            connect_reuse_seed(subseed, reuse_subseed, generation_info, dummy_component, is_subseed=True)
 
             txt2img_args = dict(
                 fn=txt2img,
@@ -310,6 +445,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     txt2img_prompt,
                     txt2img_negative_prompt,
                     txt2img_prompt_style,
+                    txt2img_prompt_style2,
                     steps,
                     sampler_index,
                     restore_faces,
@@ -321,25 +457,27 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w,
                     height,
                     width,
+                    enable_hr,
+                    scale_latent,
+                    denoising_strength,
                 ] + custom_inputs,
                 outputs=[
                     txt2img_gallery,
                     generation_info,
                     html_info
-                ]
+                ],
+                show_progress=False,
             )
 
             txt2img_prompt.submit(**txt2img_args)
             submit.click(**txt2img_args)
 
-            check_progress.click(
-                fn=check_progress_call,
-                show_progress=False,
-                inputs=[],
-                outputs=[progressbar, txt2img_preview, txt2img_preview],
+            enable_hr.change(
+                fn=lambda x: gr_show(x),
+                inputs=[enable_hr],
+                outputs=[hr_options],
             )
 
-
             interrupt.click(
                 fn=lambda: shared.state.interrupt(),
                 inputs=[],
@@ -348,9 +486,11 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
 
             save.click(
                 fn=wrap_gradio_call(save_files),
+                _js="(x, y, z) => [x, y, selected_gallery_index()]",
                 inputs=[
                     generation_info,
                     txt2img_gallery,
+                    html_info,
                 ],
                 outputs=[
                     html_info,
@@ -370,18 +510,12 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
             )
 
     with gr.Blocks(analytics_enabled=False) as img2img_interface:
-        with gr.Row(elem_id="toprow"):
-            img2img_prompt = gr.Textbox(label="Prompt", elem_id="img2img_prompt", show_label=False, placeholder="Prompt", lines=1)
-            img2img_negative_prompt = gr.Textbox(label="Negative prompt", elem_id="img2img_negative_prompt", show_label=False, placeholder="Negative prompt", lines=1)
-            img2img_prompt_style = gr.Dropdown(label="Style", show_label=False, elem_id="style_index", choices=[k for k, v in shared.prompt_styles.items()], value=next(iter(shared.prompt_styles.keys())), visible=len(shared.prompt_styles) > 1)
-            img2img_interrogate = gr.Button('Interrogate', elem_id="img2img_interrogate", variant='primary')
-            submit = gr.Button('Generate', elem_id="img2img_generate", variant='primary')
-            check_progress = gr.Button('Check progress', elem_id="check_progress", visible=False)
+        img2img_prompt, roll, img2img_prompt_style, img2img_negative_prompt, img2img_prompt_style2, submit, img2img_interrogate, img2img_prompt_style_apply, img2img_save_style = create_toprow(is_img2img=True)
 
         with gr.Row().style(equal_height=False):
             with gr.Column(variant='panel'):
                 with gr.Group():
-                    switch_mode = gr.Radio(label='Mode', elem_id="img2img_mode", choices=['Redraw whole image', 'Inpaint a part of image', 'Loopback', 'SD upscale'], value='Redraw whole image', type="index", show_label=False)
+                    switch_mode = gr.Radio(label='Mode', elem_id="img2img_mode", choices=['Redraw whole image', 'Inpaint a part of image', 'SD upscale'], value='Redraw whole image', type="index", show_label=False)
                     init_img = gr.Image(label="Image for img2img", source="upload", interactive=True, type="pil")
                     init_img_with_mask = gr.Image(label="Image for inpainting with mask", elem_id="img2maskimg", source="upload", interactive=True, type="pil", tool="sketch", visible=False, image_mode="RGBA")
                     init_mask = gr.Image(label="Mask", source="upload", interactive=True, type="pil", visible=False)
@@ -415,22 +549,25 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 with gr.Group():
                     cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.5, label='CFG Scale', value=7.0)
                     denoising_strength = gr.Slider(minimum=0.0, maximum=1.0, step=0.01, label='Denoising strength', value=0.75)
-                    denoising_strength_change_factor = gr.Slider(minimum=0.9, maximum=1.1, step=0.01, label='Denoising strength change factor', value=1, visible=False)
 
                 with gr.Group():
                     width = gr.Slider(minimum=64, maximum=2048, step=64, label="Width", value=512)
                     height = gr.Slider(minimum=64, maximum=2048, step=64, label="Height", value=512)
 
-                seed, subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w = create_seed_inputs()
+                seed, reuse_seed, subseed, reuse_subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w = create_seed_inputs()
 
                 with gr.Group():
                     custom_inputs = modules.scripts.scripts_img2img.setup_ui(is_img2img=True)
 
             with gr.Column(variant='panel'):
+                progressbar = gr.HTML(elem_id="progressbar")
+
                 with gr.Group():
                     img2img_preview = gr.Image(elem_id='img2img_preview', visible=False)
                     img2img_gallery = gr.Gallery(label='Output', elem_id='img2img_gallery').style(grid=4)
 
+                setup_progressbar(progressbar, img2img_preview)
+
                 with gr.Group():
                     with gr.Row():
                         save = gr.Button('Save')
@@ -440,17 +577,18 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                         interrupt = gr.Button('Interrupt')
                         img2img_save_style = gr.Button('Save prompt as style')
 
-                progressbar = gr.HTML(elem_id="progressbar")
 
                 with gr.Group():
                     html_info = gr.HTML()
                     generation_info = gr.Textbox(visible=False)
 
+            connect_reuse_seed(seed, reuse_seed, generation_info, dummy_component, is_subseed=False)
+            connect_reuse_seed(subseed, reuse_subseed, generation_info, dummy_component, is_subseed=True)
+
             def apply_mode(mode, uploadmask):
                 is_classic = mode == 0
                 is_inpaint = mode == 1
-                is_loopback = mode == 2
-                is_upscale = mode == 3
+                is_upscale = mode == 2
 
                 return {
                     init_img: gr_show(not is_inpaint or (is_inpaint and uploadmask == 1)),
@@ -460,12 +598,10 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     mask_mode: gr_show(is_inpaint),
                     mask_blur: gr_show(is_inpaint),
                     inpainting_fill: gr_show(is_inpaint),
-                    batch_size: gr_show(not is_loopback),
                     sd_upscale_upscaler_name: gr_show(is_upscale),
                     sd_upscale_overlap: gr_show(is_upscale),
                     inpaint_full_res: gr_show(is_inpaint),
                     inpainting_mask_invert: gr_show(is_inpaint),
-                    denoising_strength_change_factor: gr_show(is_loopback),
                     img2img_interrogate: gr_show(not is_inpaint),
                 }
 
@@ -480,12 +616,10 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     mask_mode,
                     mask_blur,
                     inpainting_fill,
-                    batch_size,
                     sd_upscale_upscaler_name,
                     sd_upscale_overlap,
                     inpaint_full_res,
                     inpainting_mask_invert,
-                    denoising_strength_change_factor,
                     img2img_interrogate,
                 ]
             )
@@ -511,6 +645,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     img2img_prompt,
                     img2img_negative_prompt,
                     img2img_prompt_style,
+                    img2img_prompt_style2,
                     init_img,
                     init_img_with_mask,
                     init_mask,
@@ -526,7 +661,6 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     batch_size,
                     cfg_scale,
                     denoising_strength,
-                    denoising_strength_change_factor,
                     seed,
                     subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w,
                     height,
@@ -541,7 +675,8 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                     img2img_gallery,
                     generation_info,
                     html_info
-                ]
+                ],
+                show_progress=False,
             )
 
             img2img_prompt.submit(**img2img_args)
@@ -553,13 +688,6 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 outputs=[img2img_prompt],
             )
 
-            check_progress.click(
-                fn=check_progress_call,
-                show_progress=False,
-                inputs=[],
-                outputs=[progressbar, img2img_preview, img2img_preview],
-            )
-
             interrupt.click(
                 fn=lambda: shared.state.interrupt(),
                 inputs=[],
@@ -568,9 +696,11 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
 
             save.click(
                 fn=wrap_gradio_call(save_files),
+                _js="(x, y, z) => [x, y, selected_gallery_index()]",
                 inputs=[
                     generation_info,
                     img2img_gallery,
+                    html_info
                 ],
                 outputs=[
                     html_info,
@@ -579,22 +709,45 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 ]
             )
 
-            dummy_component = gr.Label(visible=False)
-            for button, (prompt, negative_prompt) in zip([txt2img_save_style, img2img_save_style], [(txt2img_prompt, txt2img_negative_prompt), (img2img_prompt, img2img_negative_prompt)]):
+            roll.click(
+                fn=roll_artist,
+                inputs=[
+                    img2img_prompt,
+                ],
+                outputs=[
+                    img2img_prompt,
+                ]
+            )
+
+            prompts = [(txt2img_prompt, txt2img_negative_prompt), (img2img_prompt, img2img_negative_prompt)]
+            style_dropdowns = [(txt2img_prompt_style, txt2img_prompt_style2), (img2img_prompt_style, img2img_prompt_style2)]
+
+            for button, (prompt, negative_prompt) in zip([txt2img_save_style, img2img_save_style], prompts):
                 button.click(
                     fn=add_style,
                     _js="ask_for_style_name",
                     # Have to pass empty dummy component here, because the JavaScript and Python function have to accept
                     # the same number of parameters, but we only know the style-name after the JavaScript prompt
                     inputs=[dummy_component, prompt, negative_prompt],
-                    outputs=[txt2img_prompt_style, img2img_prompt_style],
+                    outputs=[txt2img_prompt_style, img2img_prompt_style, txt2img_prompt_style2, img2img_prompt_style2],
+                )
+
+            for button, (prompt, negative_prompt), (style1, style2) in zip([txt2img_prompt_style_apply, img2img_prompt_style_apply], prompts, style_dropdowns):
+                button.click(
+                    fn=apply_styles,
+                    inputs=[prompt, negative_prompt, style1, style2],
+                    outputs=[prompt, negative_prompt, style1, style2],
                 )
 
     with gr.Blocks(analytics_enabled=False) as extras_interface:
         with gr.Row().style(equal_height=False):
             with gr.Column(variant='panel'):
-                with gr.Group():
-                    image = gr.Image(label="Source", source="upload", interactive=True, type="pil")
+                with gr.Tabs():
+                    with gr.TabItem('Single Image'):
+                        image = gr.Image(label="Source", source="upload", interactive=True, type="pil")
+
+                    with gr.TabItem('Batch Process'):
+                        image_batch = gr.File(label="Batch Process", file_count="multiple", interactive=True, type="file")
 
                 upscaling_resize = gr.Slider(minimum=1.0, maximum=4.0, step=0.05, label="Resize", value=2)
 
@@ -615,7 +768,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 submit = gr.Button('Generate', elem_id="extras_generate", variant='primary')
 
             with gr.Column(variant='panel'):
-                result_image = gr.Image(label="Result")
+                result_images = gr.Gallery(label="Result")
                 html_info_x = gr.HTML()
                 html_info = gr.HTML()
 
@@ -623,6 +776,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
             fn=run_extras,
             inputs=[
                 image,
+                image_batch,
                 gfpgan_visibility,
                 codeformer_visibility,
                 codeformer_weight,
@@ -632,7 +786,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                 extras_upscaler_2_visibility,
             ],
             outputs=[
-                result_image,
+                result_images,
                 html_info_x,
                 html_info,
             ]
@@ -689,7 +843,12 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
             if comp_args and isinstance(comp_args, dict) and comp_args.get('visible') is False:
                 continue
 
+            oldval = opts.data.get(key, None)
             opts.data[key] = value
+
+            if oldval != value and opts.data_labels[key].onchange is not None:
+                opts.data_labels[key].onchange()
+
             up.append(comp.update(value=value))
 
         opts.save(shared.config_filename)
@@ -697,7 +856,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
         return 'Settings applied.'
 
     with gr.Blocks(analytics_enabled=False) as settings_interface:
-        submit = gr.Button(value="Apply settings", variant='primary')
+        settings_submit = gr.Button(value="Apply settings", variant='primary')
         result = gr.HTML()
 
         with gr.Row(elem_id="settings").style(equal_height=False):
@@ -709,7 +868,7 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
                         if index < len(keys):
                             components.append(create_setting_component(keys[index]))
 
-        submit.click(
+        settings_submit.click(
             fn=run_settings,
             inputs=components,
             outputs=[result]
@@ -726,15 +885,29 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
     with open(os.path.join(script_path, "style.css"), "r", encoding="utf8") as file:
         css = file.read()
 
+    if os.path.exists(os.path.join(script_path, "user.css")):
+        with open(os.path.join(script_path, "user.css"), "r", encoding="utf8") as file:
+            usercss = file.read()
+            css += usercss
+
     if not cmd_opts.no_progressbar_hiding:
         css += css_hide_progressbar
 
     with gr.Blocks(css=css, analytics_enabled=False, title="Stable Diffusion") as demo:
+
         with gr.Tabs() as tabs:
             for interface, label, ifid in interfaces:
                 with gr.TabItem(label, id=ifid):
                     interface.render()
 
+        text_settings = gr.Textbox(elem_id="settings_json", value=opts.dumpjson(), visible=False)
+
+        settings_submit.click(
+            fn=lambda: opts.dumpjson(),
+            inputs=[],
+            outputs=[text_settings],
+        )
+
         tabs.change(
             fn=lambda x: x,
             inputs=[init_img_with_mask],
@@ -828,12 +1001,17 @@ def create_ui(txt2img, img2img, run_extras, run_pnginfo):
 
 
 with open(os.path.join(script_path, "script.js"), "r", encoding="utf8") as jsfile:
-    javascript = jsfile.read()
+    javascript = f'<script>{jsfile.read()}</script>'
+
+jsdir = os.path.join(script_path, "javascript")
+for filename in os.listdir(jsdir):
+    with open(os.path.join(jsdir, filename), "r", encoding="utf8") as jsfile:
+        javascript += f"\n<script>{jsfile.read()}</script>"
 
 
 def template_response(*args, **kwargs):
     res = gradio_routes_templates_response(*args, **kwargs)
-    res.body = res.body.replace(b'</head>', f'<script>{javascript}</script></head>'.encode("utf8"))
+    res.body = res.body.replace(b'</head>', f'{javascript}</head>'.encode("utf8"))
     res.init_headers()
     return res
 

requirements_versions.txt

@@ -2,7 +2,7 @@ transformers==4.19.2
 diffusers==0.2.4
 basicsr==1.3.5
 gfpgan
-gradio==3.3
+gradio==3.3.1
 numpy==1.23.3
 Pillow==9.2.0
 realesrgan==0.2.5.0

script.js

@@ -1,197 +1,25 @@
-
-
-titles = {
-    "Sampling steps": "How many times to improve the generated image iteratively; higher values take longer; very low values can produce bad results",
-    "Sampling method": "Which algorithm to use to produce the image",
-	"GFPGAN": "Restore low quality faces using GFPGAN neural network",
-	"Euler a": "Euler Ancestral - very creative, each can get a completely different picture depending on step count, setting steps to higher than 30-40 does not help",
-	"DDIM": "Denoising Diffusion Implicit Models - best at inpainting",
-
-	"Batch count": "How many batches of images to create",
-	"Batch size": "How many image to create in a single batch",
-    "CFG Scale": "Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results",
-    "Seed": "A value that determines the output of random number generator - if you create an image with same parameters and seed as another image, you'll get the same result",
-
-    "Inpaint a part of image": "Draw a mask over an image, and the script will regenerate the masked area with content according to prompt",
-    "Loopback": "Process an image, use it as an input, repeat. Batch count determins number of iterations.",
-    "SD upscale": "Upscale image normally, split result into tiles, improve each tile using img2img, merge whole image back",
-
-    "Just resize": "Resize image to target resolution. Unless height and width match, you will get incorrect aspect ratio.",
-    "Crop and resize": "Resize the image so that entirety of target resolution is filled with the image. Crop parts that stick out.",
-    "Resize and fill": "Resize the image so that entirety of image is inside target resolution. Fill empty space with image's colors.",
-
-    "Mask blur": "How much to blur the mask before processing, in pixels.",
-    "Masked content": "What to put inside the masked area before processing it with Stable Diffusion.",
-    "fill": "fill it with colors of the image",
-    "original": "keep whatever was there originally",
-    "latent noise": "fill it with latent space noise",
-    "latent nothing": "fill it with latent space zeroes",
-    "Inpaint at full resolution": "Upscale masked region to target resolution, do inpainting, downscale back and paste into original image",
-
-    "Denoising strength": "Determines how little respect the algorithm should have for image's content. At 0, nothing will change, and at 1 you'll get an unrelated image. With values below 1.0, processing will take less steps than the Sampling Steps slider specifies.",
-    "Denoising strength change factor": "In loopback mode, on each loop the denoising strength is multiplied by this value. <1 means decreasing variety so your sequence will converge on a fixed picture. >1 means increasing variety so your sequence will become more and more chaotic.",
-
-    "Interrupt": "Stop processing images and return any results accumulated so far.",
-    "Save": "Write image to a directory (default - log/images) and generation parameters into csv file.",
-
-    "X values": "Separate values for X axis using commas.",
-    "Y values": "Separate values for Y axis using commas.",
-
-    "None": "Do not do anything special",
-    "Prompt matrix": "Separate prompts into parts using vertical pipe character (|) and the script will create a picture for every combination of them (except for the first part, which will be present in all combinations)",
-    "X/Y plot": "Create a grid where images will have different parameters. Use inputs below to specify which parameters will be shared by columns and rows",
-    "Custom code": "Run Python code. Advanced user only. Must run program with --allow-code for this to work",
-
-    "Prompt S/R": "Separate a list of words with commas, and the first word will be used as a keyword: script will search for this word in the prompt, and replace it with others",
-
-    "Tiling": "Produce an image that can be tiled.",
-    "Tile overlap": "For SD upscale, how much overlap in pixels should there be between tiles. Tiles overlap so that when they are merged back into one picture, there is no clearly visible seam.",
-
-    "Roll": "Add a random artist to the prompt.",
-
-    "Variation seed": "Seed of a different picture to be mixed into the generation.",
-    "Variation strength": "How strong of a variation to produce. At 0, there will be no effect. At 1, you will get the complete picture with variation seed (except for ancestral samplers, where you will just get something).",
-    "Resize seed from height": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution",
-    "Resize seed from width": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution",
-
-    "Interrogate": "Reconstruct frompt from existing image and put it into the prompt field.",
-
-    "Images filename pattern": "Use following tags to define how filenames for images are chosen: [steps], [cfg], [prompt], [prompt_spaces], [width], [height], [sampler], [seed], [model_hash], [prompt_words], [date]; leave empty for default.",
-    "Directory name pattern": "Use following tags to define how subdirectories for images and grids are chosen: [steps], [cfg], [prompt], [prompt_spaces], [width], [height], [sampler], [seed], [model_hash], [prompt_words], [date]; leave empty for default.",
-}
-
 function gradioApp(){
     return document.getElementsByTagName('gradio-app')[0].shadowRoot;
 }
 
-global_progressbar = null
+uiUpdateCallbacks = []
+function onUiUpdate(callback){
+    uiUpdateCallbacks.push(callback)
+}
 
-function addTitles(root){
-	root.querySelectorAll('span, button, select').forEach(function(span){
-		tooltip = titles[span.textContent];
-
-		if(!tooltip){
-		    tooltip = titles[span.value];
-		}
-
-		if(tooltip){
-			span.title = tooltip;
-		}
+function uiUpdate(root){
+	uiUpdateCallbacks.forEach(function(x){
+        try {
+            x()
+        } catch (e) {
+            (console.error || console.log).call(console, e.message, e);
+        }
 	})
-
-	root.querySelectorAll('select').forEach(function(select){
-	    if (select.onchange != null) return;
-
-	    select.onchange = function(){
-            select.title = titles[select.value] || "";
-	    }
-	})
-
-	progressbar = root.getElementById('progressbar')
-	if(progressbar!= null && progressbar != global_progressbar){
-	    global_progressbar = progressbar
-
-        var mutationObserver = new MutationObserver(function(m){
-            txt2img_preview = gradioApp().getElementById('txt2img_preview')
-            txt2img_gallery = gradioApp().getElementById('txt2img_gallery')
-
-            img2img_preview = gradioApp().getElementById('img2img_preview')
-            img2img_gallery = gradioApp().getElementById('img2img_gallery')
-
-            if(txt2img_preview != null && txt2img_gallery != null){
-                txt2img_preview.style.width = txt2img_gallery.clientWidth + "px"
-                txt2img_preview.style.height = txt2img_gallery.clientHeight + "px"
-            }
-
-            if(img2img_preview != null && img2img_gallery != null){
-                img2img_preview.style.width = img2img_gallery.clientWidth + "px"
-                img2img_preview.style.height = img2img_gallery.clientHeight + "px"
-            }
-
-
-            window.setTimeout(requestProgress, 500)
-        });
-        mutationObserver.observe( progressbar, { childList:true, subtree:true })
-	}
-
 }
 
 document.addEventListener("DOMContentLoaded", function() {
     var mutationObserver = new MutationObserver(function(m){
-        addTitles(gradioApp());
+        uiUpdate(gradioApp());
     });
     mutationObserver.observe( gradioApp(), { childList:true, subtree:true })
 });
-
-function selected_gallery_index(){
-    var gr = gradioApp()
-    var buttons = gradioApp().querySelectorAll(".gallery-item")
-    var button = gr.querySelector(".gallery-item.\\!ring-2")
-
-    var result = -1
-    buttons.forEach(function(v, i){ if(v==button) { result = i } })
-
-    return result
-}
-
-function extract_image_from_gallery(gallery){
-    if(gallery.length == 1){
-        return gallery[0]
-    }
-
-    index = selected_gallery_index()
-
-    if (index < 0 || index >= gallery.length){
-        return [null]
-    }
-
-    return gallery[index];
-}
-
-function extract_image_from_gallery_img2img(gallery){
-    gradioApp().querySelectorAll('button')[1].click();
-    return extract_image_from_gallery(gallery);
-}
-
-function extract_image_from_gallery_extras(gallery){
-    gradioApp().querySelectorAll('button')[2].click();
-    return extract_image_from_gallery(gallery);
-}
-
-function requestProgress(){
-    btn = gradioApp().getElementById("check_progress");
-    if(btn==null) return;
-
-    btn.click();
-}
-
-function submit(){
-    window.setTimeout(requestProgress, 500)
-
-    res = []
-    for(var i=0;i<arguments.length;i++){
-        res.push(arguments[i])
-    }
-    return res
-}
-
-window.addEventListener('paste', e => {
-    const files = e.clipboardData.files;
-    if (!files || files.length !== 1) {
-        return;
-    }
-    if (!['image/png', 'image/gif', 'image/jpeg'].includes(files[0].type)) {
-        return;
-    }
-    [...gradioApp().querySelectorAll('input[type=file][accept="image/x-png,image/gif,image/jpeg"]')]
-        .filter(input => !input.matches('.\\!hidden input[type=file]'))
-        .forEach(input => {
-            input.files = files;
-            input.dispatchEvent(new Event('change'))
-        });
-});
-
-function ask_for_style_name(_, prompt_text, negative_prompt_text) {
-    name_ = prompt('Style name:')
-    return name_ === null ? [null, null, null]: [name_, prompt_text, negative_prompt_text]
-}

scripts/custom_code.py

@@ -4,8 +4,8 @@ import gradio as gr
 from modules.processing import Processed
 from modules.shared import opts, cmd_opts, state
 
-
 class Script(scripts.Script):
+
     def title(self):
         return "Custom code"
 
@@ -18,6 +18,7 @@ class Script(scripts.Script):
 
         return [code]
 
+
     def run(self, p, code):
         assert cmd_opts.allow_code, '--allow-code option must be enabled'
 
@@ -37,4 +38,5 @@ class Script(scripts.Script):
         exec(compiled, module.__dict__)
 
         return Processed(p, *display_result_data)
-
+    
+    

scripts/img2imgalt.py

@@ -1,10 +1,12 @@
+from collections import namedtuple
+
 import numpy as np
 from tqdm import trange
 
 import modules.scripts as scripts
 import gradio as gr
 
-from modules import processing, shared, sd_samplers
+from modules import processing, shared, sd_samplers, prompt_parser
 from modules.processing import Processed
 from modules.sd_samplers import samplers
 from modules.shared import opts, cmd_opts, state
@@ -56,9 +58,14 @@ def find_noise_for_image(p, cond, uncond, cfg_scale, steps):
 
     return x / x.std()
 
-cache = [None, None, None, None, None]
+
+Cached = namedtuple("Cached", ["noise", "cfg_scale", "steps", "latent", "original_prompt", "original_negative_prompt"])
+
 
 class Script(scripts.Script):
+    def __init__(self):
+        self.cache = None
+
     def title(self):
         return "img2img alternative test"
 
@@ -67,37 +74,54 @@ class Script(scripts.Script):
 
     def ui(self, is_img2img):
         original_prompt = gr.Textbox(label="Original prompt", lines=1)
-        cfg = gr.Slider(label="Decode CFG scale", minimum=0.1, maximum=3.0, step=0.1, value=1.0)
+        original_negative_prompt = gr.Textbox(label="Original negative prompt", lines=1)
+        cfg = gr.Slider(label="Decode CFG scale", minimum=0.0, maximum=15.0, step=0.1, value=1.0)
         st = gr.Slider(label="Decode steps", minimum=1, maximum=150, step=1, value=50)
+        randomness = gr.Slider(label="randomness", minimum=0.0, maximum=1.0, step=0.01, value=0.0)
+        return [original_prompt, original_negative_prompt, cfg, st, randomness]
 
-        return [original_prompt, cfg, st]
-
-    def run(self, p, original_prompt, cfg, st):
+    def run(self, p, original_prompt, original_negative_prompt, cfg, st, randomness):
         p.batch_size = 1
         p.batch_count = 1
 
         def sample_extra(x, conditioning, unconditional_conditioning):
-            lat = tuple([int(x*10) for x in p.init_latent.cpu().numpy().flatten().tolist()])
+            lat = (p.init_latent.cpu().numpy() * 10).astype(int)
 
-            if cache[0] is not None and cache[1] == cfg and cache[2] == st and len(cache[3]) == len(lat) and sum(np.array(cache[3])-np.array(lat)) < 100 and cache[4] == original_prompt:
-                noise = cache[0]
+            same_params = self.cache is not None and self.cache.cfg_scale == cfg and self.cache.steps == st and self.cache.original_prompt == original_prompt and self.cache.original_negative_prompt == original_negative_prompt
+            same_everything = same_params and self.cache.latent.shape == lat.shape and np.abs(self.cache.latent-lat).sum() < 100
+
+            if same_everything:
+                rec_noise = self.cache.noise
             else:
                 shared.state.job_count += 1
                 cond = p.sd_model.get_learned_conditioning(p.batch_size * [original_prompt])
-                noise = find_noise_for_image(p, cond, unconditional_conditioning, cfg, st)
-                cache[0] = noise
-                cache[1] = cfg
-                cache[2] = st
-                cache[3] = lat
-                cache[4] = original_prompt
+                uncond = p.sd_model.get_learned_conditioning(p.batch_size * [original_negative_prompt])
+                rec_noise = find_noise_for_image(p, cond, uncond, cfg, st)
+                self.cache = Cached(rec_noise, cfg, st, lat, original_prompt, original_negative_prompt)
 
+            rand_noise = processing.create_random_tensors(p.init_latent.shape[1:], [p.seed + x + 1 for x in range(p.init_latent.shape[0])])
+            
+            combined_noise = ((1 - randomness) * rec_noise + randomness * rand_noise) / ((randomness**2 + (1-randomness)**2) ** 0.5)
+            
             sampler = samplers[p.sampler_index].constructor(p.sd_model)
 
-            samples_ddim = sampler.sample(p, noise, conditioning, unconditional_conditioning)
-            return samples_ddim
+            sigmas = sampler.model_wrap.get_sigmas(p.steps)
+            
+            noise_dt = combined_noise - ( p.init_latent / sigmas[0] )
+            
+            p.seed = p.seed + 1
+            
+            return sampler.sample_img2img(p, p.init_latent, noise_dt, conditioning, unconditional_conditioning)
+
 
         p.sample = sample_extra
 
+        p.extra_generation_params = {
+            "Decode prompt": original_prompt,
+            "Decode CFG scale": cfg,
+            "Decode steps": st,
+        }
+
         processed = processing.process_images(p)
 
         return processed

scripts/loopback.py

@@ -0,0 +1,83 @@
+import numpy as np
+from tqdm import trange
+
+import modules.scripts as scripts
+import gradio as gr
+
+from modules import processing, shared, sd_samplers, images
+from modules.processing import Processed
+from modules.sd_samplers import samplers
+from modules.shared import opts, cmd_opts, state
+
+class Script(scripts.Script):
+    def title(self):
+        return "Loopback"
+
+    def show(self, is_img2img):
+        return is_img2img
+
+    def ui(self, is_img2img):
+        loops = gr.Slider(minimum=1, maximum=32, step=1, label='Loops', value=4)
+        denoising_strength_change_factor = gr.Slider(minimum=0.9, maximum=1.1, step=0.01, label='Denoising strength change factor', value=1)
+
+        return [loops, denoising_strength_change_factor]
+
+    def run(self, p, loops, denoising_strength_change_factor):
+        processing.fix_seed(p)
+        batch_count = p.n_iter
+        p.extra_generation_params = {
+            "Denoising strength change factor": denoising_strength_change_factor,
+        }
+
+        p.batch_size = 1
+        p.n_iter = 1
+
+        output_images, info = None, None
+        initial_seed = None
+        initial_info = None
+
+        grids = []
+        all_images = []
+        state.job_count = loops * batch_count
+
+        initial_color_corrections = [processing.setup_color_correction(p.init_images[0])]
+
+        for n in range(batch_count):
+            history = []
+
+            for i in range(loops):
+                p.n_iter = 1
+                p.batch_size = 1
+                p.do_not_save_grid = True
+
+                if opts.img2img_color_correction:
+                    p.color_corrections = initial_color_corrections
+
+                state.job = f"Iteration {i + 1}/{loops}, batch {n + 1}/{batch_count}"
+
+                processed = processing.process_images(p)
+
+                if initial_seed is None:
+                    initial_seed = processed.seed
+                    initial_info = processed.info
+
+                init_img = processed.images[0]
+
+                p.init_images = [init_img]
+                p.seed = processed.seed + 1
+                p.denoising_strength = min(max(p.denoising_strength * denoising_strength_change_factor, 0.1), 1)
+                history.append(processed.images[0])
+
+            grid = images.image_grid(history, rows=1)
+            if opts.grid_save:
+                images.save_image(grid, p.outpath_grids, "grid", initial_seed, p.prompt, opts.grid_format, info=info, short_filename=not opts.grid_extended_filename, grid=True, p=p)
+
+            grids.append(grid)
+            all_images += history
+
+        if opts.return_grid:
+            all_images = grids + all_images
+
+        processed = Processed(p, all_images, initial_seed, initial_info)
+
+        return processed

scripts/outpainting_mk_2.py

@@ -0,0 +1,290 @@
+import math
+
+import numpy as np
+import skimage
+
+import modules.scripts as scripts
+import gradio as gr
+from PIL import Image, ImageDraw
+
+from modules import images, processing, devices
+from modules.processing import Processed, process_images
+from modules.shared import opts, cmd_opts, state
+
+
+def expand(x, dir, amount, power=0.75):
+    is_left = dir == 3
+    is_right = dir == 1
+    is_up = dir == 0
+    is_down = dir == 2
+
+    if is_left or is_right:
+        noise = np.zeros((x.shape[0], amount, 3), dtype=float)
+        indexes = np.random.random((x.shape[0], amount)) ** power * (1 - np.arange(amount) / amount)
+        if is_right:
+            indexes = 1 - indexes
+        indexes = (indexes * (x.shape[1] - 1)).astype(int)
+
+        for row in range(x.shape[0]):
+            if is_left:
+                noise[row] = x[row][indexes[row]]
+            else:
+                noise[row] = np.flip(x[row][indexes[row]], axis=0)
+
+        x = np.concatenate([noise, x] if is_left else [x, noise], axis=1)
+        return x
+
+    if is_up or is_down:
+        noise = np.zeros((amount, x.shape[1], 3), dtype=float)
+        indexes = np.random.random((x.shape[1], amount)) ** power * (1 - np.arange(amount) / amount)
+        if is_down:
+            indexes = 1 - indexes
+        indexes = (indexes * x.shape[0] - 1).astype(int)
+
+        for row in range(x.shape[1]):
+            if is_up:
+                noise[:, row] = x[:, row][indexes[row]]
+            else:
+                noise[:, row] = np.flip(x[:, row][indexes[row]], axis=0)
+
+        x = np.concatenate([noise, x] if is_up else [x, noise], axis=0)
+        return x
+
+
+def get_matched_noise(_np_src_image, np_mask_rgb, noise_q=1, color_variation=0.05):
+    # helper fft routines that keep ortho normalization and auto-shift before and after fft
+    def _fft2(data):
+        if data.ndim > 2:  # has channels
+            out_fft = np.zeros((data.shape[0], data.shape[1], data.shape[2]), dtype=np.complex128)
+            for c in range(data.shape[2]):
+                c_data = data[:, :, c]
+                out_fft[:, :, c] = np.fft.fft2(np.fft.fftshift(c_data), norm="ortho")
+                out_fft[:, :, c] = np.fft.ifftshift(out_fft[:, :, c])
+        else:  # one channel
+            out_fft = np.zeros((data.shape[0], data.shape[1]), dtype=np.complex128)
+            out_fft[:, :] = np.fft.fft2(np.fft.fftshift(data), norm="ortho")
+            out_fft[:, :] = np.fft.ifftshift(out_fft[:, :])
+
+        return out_fft
+
+    def _ifft2(data):
+        if data.ndim > 2:  # has channels
+            out_ifft = np.zeros((data.shape[0], data.shape[1], data.shape[2]), dtype=np.complex128)
+            for c in range(data.shape[2]):
+                c_data = data[:, :, c]
+                out_ifft[:, :, c] = np.fft.ifft2(np.fft.fftshift(c_data), norm="ortho")
+                out_ifft[:, :, c] = np.fft.ifftshift(out_ifft[:, :, c])
+        else:  # one channel
+            out_ifft = np.zeros((data.shape[0], data.shape[1]), dtype=np.complex128)
+            out_ifft[:, :] = np.fft.ifft2(np.fft.fftshift(data), norm="ortho")
+            out_ifft[:, :] = np.fft.ifftshift(out_ifft[:, :])
+
+        return out_ifft
+
+    def _get_gaussian_window(width, height, std=3.14, mode=0):
+        window_scale_x = float(width / min(width, height))
+        window_scale_y = float(height / min(width, height))
+
+        window = np.zeros((width, height))
+        x = (np.arange(width) / width * 2. - 1.) * window_scale_x
+        for y in range(height):
+            fy = (y / height * 2. - 1.) * window_scale_y
+            if mode == 0:
+                window[:, y] = np.exp(-(x ** 2 + fy ** 2) * std)
+            else:
+                window[:, y] = (1 / ((x ** 2 + 1.) * (fy ** 2 + 1.))) ** (std / 3.14)  # hey wait a minute that's not gaussian
+
+        return window
+
+    def _get_masked_window_rgb(np_mask_grey, hardness=1.):
+        np_mask_rgb = np.zeros((np_mask_grey.shape[0], np_mask_grey.shape[1], 3))
+        if hardness != 1.:
+            hardened = np_mask_grey[:] ** hardness
+        else:
+            hardened = np_mask_grey[:]
+        for c in range(3):
+            np_mask_rgb[:, :, c] = hardened[:]
+        return np_mask_rgb
+
+    width = _np_src_image.shape[0]
+    height = _np_src_image.shape[1]
+    num_channels = _np_src_image.shape[2]
+
+    np_src_image = _np_src_image[:] * (1. - np_mask_rgb)
+    np_mask_grey = (np.sum(np_mask_rgb, axis=2) / 3.)
+    img_mask = np_mask_grey > 1e-6
+    ref_mask = np_mask_grey < 1e-3
+
+    windowed_image = _np_src_image * (1. - _get_masked_window_rgb(np_mask_grey))
+    windowed_image /= np.max(windowed_image)
+    windowed_image += np.average(_np_src_image) * np_mask_rgb  # / (1.-np.average(np_mask_rgb))  # rather than leave the masked area black, we get better results from fft by filling the average unmasked color
+
+    src_fft = _fft2(windowed_image)  # get feature statistics from masked src img
+    src_dist = np.absolute(src_fft)
+    src_phase = src_fft / src_dist
+
+    noise_window = _get_gaussian_window(width, height, mode=1)  # start with simple gaussian noise
+    noise_rgb = np.random.random_sample((width, height, num_channels))
+    noise_grey = (np.sum(noise_rgb, axis=2) / 3.)
+    noise_rgb *= color_variation  # the colorfulness of the starting noise is blended to greyscale with a parameter
+    for c in range(num_channels):
+        noise_rgb[:, :, c] += (1. - color_variation) * noise_grey
+
+    noise_fft = _fft2(noise_rgb)
+    for c in range(num_channels):
+        noise_fft[:, :, c] *= noise_window
+    noise_rgb = np.real(_ifft2(noise_fft))
+    shaped_noise_fft = _fft2(noise_rgb)
+    shaped_noise_fft[:, :, :] = np.absolute(shaped_noise_fft[:, :, :]) ** 2 * (src_dist ** noise_q) * src_phase  # perform the actual shaping
+
+    brightness_variation = 0.  # color_variation # todo: temporarily tieing brightness variation to color variation for now
+    contrast_adjusted_np_src = _np_src_image[:] * (brightness_variation + 1.) - brightness_variation * 2.
+
+    # scikit-image is used for histogram matching, very convenient!
+    shaped_noise = np.real(_ifft2(shaped_noise_fft))
+    shaped_noise -= np.min(shaped_noise)
+    shaped_noise /= np.max(shaped_noise)
+    shaped_noise[img_mask, :] = skimage.exposure.match_histograms(shaped_noise[img_mask, :] ** 1., contrast_adjusted_np_src[ref_mask, :], channel_axis=1)
+    shaped_noise = _np_src_image[:] * (1. - np_mask_rgb) + shaped_noise * np_mask_rgb
+
+    matched_noise = shaped_noise[:]
+
+    return np.clip(matched_noise, 0., 1.)
+
+
+
+class Script(scripts.Script):
+    def title(self):
+        return "Outpainting mk2"
+
+    def show(self, is_img2img):
+        return is_img2img
+
+    def ui(self, is_img2img):
+        if not is_img2img:
+            return None
+
+        info = gr.HTML("<p style=\"margin-bottom:0.75em\">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>")
+
+        pixels = gr.Slider(label="Pixels to expand", minimum=8, maximum=256, step=8, value=128)
+        mask_blur = gr.Slider(label='Mask blur', minimum=0, maximum=64, step=1, value=8, visible=False)
+        direction = gr.CheckboxGroup(label="Outpainting direction", choices=['left', 'right', 'up', 'down'], value=['left', 'right', 'up', 'down'])
+        noise_q = gr.Slider(label="Fall-off exponent (lower=higher detail)", minimum=0.0, maximum=4.0, step=0.01, value=1.0)
+        color_variation = gr.Slider(label="Color variation", minimum=0.0, maximum=1.0, step=0.01, value=0.05)
+
+        return [info, pixels, mask_blur, direction, noise_q, color_variation]
+
+    def run(self, p, _, pixels, mask_blur, direction, noise_q, color_variation):
+        initial_seed_and_info = [None, None]
+
+        process_width = p.width
+        process_height = p.height
+
+        p.mask_blur = mask_blur*4
+        p.inpaint_full_res = False
+        p.inpainting_fill = 1
+        p.do_not_save_samples = True
+        p.do_not_save_grid = True
+
+        left = pixels if "left" in direction else 0
+        right = pixels if "right" in direction else 0
+        up = pixels if "up" in direction else 0
+        down = pixels if "down" in direction else 0
+
+        init_img = p.init_images[0]
+        target_w = math.ceil((init_img.width + left + right) / 64) * 64
+        target_h = math.ceil((init_img.height + up + down) / 64) * 64
+
+        if left > 0:
+            left = left * (target_w - init_img.width) // (left + right)
+        if right > 0:
+            right = target_w - init_img.width - left
+
+        if up > 0:
+            up = up * (target_h - init_img.height) // (up + down)
+
+        if down > 0:
+            down = target_h - init_img.height - up
+
+        init_image = p.init_images[0]
+
+        state.job_count = (1 if left > 0 else 0) + (1 if right > 0 else 0)+ (1 if up > 0 else 0)+ (1 if down > 0 else 0)
+
+        def expand(init, expand_pixels, is_left=False, is_right=False, is_top=False, is_bottom=False):
+            is_horiz = is_left or is_right
+            is_vert = is_top or is_bottom
+            pixels_horiz = expand_pixels if is_horiz else 0
+            pixels_vert = expand_pixels if is_vert else 0
+
+            img = Image.new("RGB", (init.width + pixels_horiz, init.height + pixels_vert))
+            img.paste(init, (pixels_horiz if is_left else 0, pixels_vert if is_top else 0))
+            mask = Image.new("RGB", (init.width + pixels_horiz, init.height + pixels_vert), "white")
+            draw = ImageDraw.Draw(mask)
+            draw.rectangle((
+                expand_pixels + mask_blur if is_left else 0,
+                expand_pixels + mask_blur if is_top else 0,
+                mask.width - expand_pixels - mask_blur if is_right else mask.width,
+                mask.height - expand_pixels - mask_blur if is_bottom else mask.height,
+            ), fill="black")
+
+            np_image = (np.asarray(img) / 255.0).astype(np.float64)
+            np_mask = (np.asarray(mask) / 255.0).astype(np.float64)
+            noised = get_matched_noise(np_image, np_mask, noise_q, color_variation)
+            out = Image.fromarray(np.clip(noised * 255., 0., 255.).astype(np.uint8), mode="RGB")
+
+            target_width = min(process_width, init.width + pixels_horiz) if is_horiz else img.width
+            target_height = min(process_height, init.height + pixels_vert) if is_vert else img.height
+
+            crop_region = (
+                0 if is_left else out.width - target_width,
+                0 if is_top else out.height - target_height,
+                target_width if is_left else out.width,
+                target_height if is_top else out.height,
+            )
+
+            image_to_process = out.crop(crop_region)
+            mask = mask.crop(crop_region)
+
+            p.width = target_width if is_horiz else img.width
+            p.height = target_height if is_vert else img.height
+            p.init_images = [image_to_process]
+            p.image_mask = mask
+
+            latent_mask = Image.new("RGB", (p.width, p.height), "white")
+            draw = ImageDraw.Draw(latent_mask)
+            draw.rectangle((
+                expand_pixels + mask_blur * 2 if is_left else 0,
+                expand_pixels + mask_blur * 2 if is_top else 0,
+                mask.width - expand_pixels - mask_blur * 2 if is_right else mask.width,
+                mask.height - expand_pixels - mask_blur * 2 if is_bottom else mask.height,
+            ), fill="black")
+            p.latent_mask = latent_mask
+
+            proc = process_images(p)
+            proc_img = proc.images[0]
+
+            if initial_seed_and_info[0] is None:
+                initial_seed_and_info[0] = proc.seed
+                initial_seed_and_info[1] = proc.info
+
+            out.paste(proc_img, (0 if is_left else out.width - proc_img.width, 0 if is_top else out.height - proc_img.height))
+            return out
+
+        img = init_image
+
+        if left > 0:
+            img = expand(img, left, is_left=True)
+        if right > 0:
+            img = expand(img, right, is_right=True)
+        if up > 0:
+            img = expand(img, up, is_top=True)
+        if down > 0:
+            img = expand(img, down, is_bottom=True)
+
+        res = Processed(p, [img], initial_seed_and_info[0], initial_seed_and_info[1])
+
+        if opts.samples_save:
+            images.save_image(img, p.outpath_samples, "", res.seed, p.prompt, opts.grid_format, info=res.info, p=p)
+
+        return res
+

scripts/prompt_matrix.py

@@ -82,6 +82,6 @@ class Script(scripts.Script):
         processed.images.insert(0, grid)
 
         if opts.grid_save:
-            images.save_image(processed.images[0], p.outpath_grids, "prompt_matrix", prompt=original_prompt, seed=processed.seed, p=p)
+            images.save_image(processed.images[0], p.outpath_grids, "prompt_matrix", prompt=original_prompt, seed=processed.seed, grid=True, p=p)
 
         return processed

scripts/prompts_from_file.py

@@ -13,28 +13,42 @@ from modules.shared import opts, cmd_opts, state
 
 class Script(scripts.Script):
     def title(self):
-        return "Prompts from file"
+        return "Prompts from file or textbox"
 
     def ui(self, is_img2img):
+        # This checkbox would look nicer as two tabs, but there are two problems:
+        # 1) There is a bug in Gradio 3.3 that prevents visibility from working on Tabs
+        # 2) Even with Gradio 3.3.1, returning a control (like Tabs) that can't be used as input
+        #    causes a AttributeError: 'Tabs' object has no attribute 'preprocess' assert,
+        #    due to the way Script assumes all controls returned can be used as inputs.
+        # Therefore, there's no good way to use grouping components right now,
+        # so we will use a checkbox! :)
+        checkbox_txt = gr.Checkbox(label="Show Textbox", value=False)
         file = gr.File(label="File with inputs", type='bytes')
+        prompt_txt = gr.TextArea(label="Prompts")
+        checkbox_txt.change(fn=lambda x: [gr.File.update(visible = not x), gr.TextArea.update(visible = x)], inputs=[checkbox_txt], outputs=[file, prompt_txt])
+        return [checkbox_txt, file, prompt_txt]
 
-        return [file]
-
-    def run(self, p, data: bytes):
-        lines = [x.strip() for x in data.decode('utf8', errors='ignore').split("\n")]
+    def run(self, p, checkbox_txt, data: bytes, prompt_txt: str):
+        if (checkbox_txt):
+            lines = [x.strip() for x in prompt_txt.splitlines()]
+        else:
+            lines = [x.strip() for x in data.decode('utf8', errors='ignore').split("\n")]
         lines = [x for x in lines if len(x) > 0]
 
-        batch_count = math.ceil(len(lines) / p.batch_size)
-        print(f"Will process {len(lines) * p.n_iter} images in {batch_count * p.n_iter} batches.")
+        img_count = len(lines) * p.n_iter
+        batch_count = math.ceil(img_count / p.batch_size)
+        loop_count = math.ceil(batch_count / p.n_iter)
+        print(f"Will process {img_count} images in {batch_count} batches.")
 
         p.do_not_save_grid = True
 
         state.job_count = batch_count
 
         images = []
-        for batch_no in range(batch_count):
-            state.job = f"{batch_no} out of {batch_count * p.n_iter}"
-            p.prompt = lines[batch_no*p.batch_size:(batch_no+1)*p.batch_size] * p.n_iter
+        for loop_no in range(loop_count):
+            state.job = f"{loop_no + 1} out of {loop_count}"
+            p.prompt = lines[loop_no*p.batch_size:(loop_no+1)*p.batch_size] * p.n_iter
             proc = process_images(p)
             images += proc.images
 

scripts/xy_grid.py

@@ -10,7 +10,9 @@ import gradio as gr
 from modules import images
 from modules.processing import process_images, Processed
 from modules.shared import opts, cmd_opts, state
+import modules.shared as shared
 import modules.sd_samplers
+import modules.sd_models
 import re
 
 
@@ -41,6 +43,15 @@ def apply_sampler(p, x, xs):
     p.sampler_index = sampler_index
 
 
+def apply_checkpoint(p, x, xs):
+    applicable = [info for info in modules.sd_models.checkpoints_list.values() if x in info.title]
+    assert len(applicable) > 0, f'Checkpoint {x} for found'
+
+    info = applicable[0]
+
+    modules.sd_models.reload_model_weights(shared.sd_model, info)
+
+
 def format_value_add_label(p, opt, x):
     if type(x) == float:
         x = round(x, 8)
@@ -74,19 +85,20 @@ axis_options = [
     AxisOption("CFG Scale", float, apply_field("cfg_scale"), format_value_add_label),
     AxisOption("Prompt S/R", str, apply_prompt, format_value),
     AxisOption("Sampler", str, apply_sampler, format_value),
+    AxisOption("Checkpoint name", str, apply_checkpoint, format_value),
     AxisOptionImg2Img("Denoising", float, apply_field("denoising_strength"), format_value_add_label), #  as it is now all AxisOptionImg2Img items must go after AxisOption ones
 ]
 
 
-def draw_xy_grid(xs, ys, x_label, y_label, cell):
+def draw_xy_grid(p, xs, ys, x_labels, y_labels, cell, draw_legend):
     res = []
 
-    ver_texts = [[images.GridAnnotation(y_label(y))] for y in ys]
-    hor_texts = [[images.GridAnnotation(x_label(x))] for x in xs]
+    ver_texts = [[images.GridAnnotation(y)] for y in y_labels]
+    hor_texts = [[images.GridAnnotation(x)] for x in x_labels]
 
     first_pocessed = None
 
-    state.job_count = len(xs) * len(ys)
+    state.job_count = len(xs) * len(ys) * p.n_iter
 
     for iy, y in enumerate(ys):
         for ix, x in enumerate(xs):
@@ -99,7 +111,8 @@ def draw_xy_grid(xs, ys, x_label, y_label, cell):
             res.append(processed.images[0])
 
     grid = images.image_grid(res, rows=len(ys))
-    grid = images.draw_grid_annotations(grid, res[0].width, res[0].height, hor_texts, ver_texts)
+    if draw_legend:
+        grid = images.draw_grid_annotations(grid, res[0].width, res[0].height, hor_texts, ver_texts)
 
     first_pocessed.images = [grid]
 
@@ -109,6 +122,9 @@ def draw_xy_grid(xs, ys, x_label, y_label, cell):
 re_range = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\(([+-]\d+)\s*\))?\s*")
 re_range_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\(([+-]\d+(?:.\d*)?)\s*\))?\s*")
 
+re_range_count = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\[(\d+)\s*\])?\s*")
+re_range_count_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\[(\d+(?:.\d*)?)\s*\])?\s*")
+
 class Script(scripts.Script):
     def title(self):
         return "X/Y plot"
@@ -123,13 +139,14 @@ class Script(scripts.Script):
         with gr.Row():
             y_type = gr.Dropdown(label="Y type", choices=[x.label for x in current_axis_options], value=current_axis_options[4].label, visible=False, type="index", elem_id="y_type")
             y_values = gr.Textbox(label="Y values", visible=False, lines=1)
+        
+        draw_legend = gr.Checkbox(label='Draw legend', value=True)
+            
+        return [x_type, x_values, y_type, y_values, draw_legend]
 
-        return [x_type, x_values, y_type, y_values]
-
-    def run(self, p, x_type, x_values, y_type, y_values):
+    def run(self, p, x_type, x_values, y_type, y_values, draw_legend):
         modules.processing.fix_seed(p)
         p.batch_size = 1
-        p.batch_count = 1
 
         def process_axis(opt, vals):
             valslist = [x.strip() for x in vals.split(",")]
@@ -139,6 +156,7 @@ class Script(scripts.Script):
 
                 for val in valslist:
                     m = re_range.fullmatch(val)
+                    mc = re_range_count.fullmatch(val)
                     if m is not None:
 
                         start = int(m.group(1))
@@ -146,6 +164,12 @@ class Script(scripts.Script):
                         step = int(m.group(3)) if m.group(3) is not None else 1
 
                         valslist_ext += list(range(start, end, step))
+                    elif mc is not None:
+                        start = int(mc.group(1))
+                        end   = int(mc.group(2))
+                        num   = int(mc.group(3)) if mc.group(3) is not None else 1
+                        
+                        valslist_ext += [int(x) for x in np.linspace(start = start, stop = end, num = num).tolist()]
                     else:
                         valslist_ext.append(val)
 
@@ -155,12 +179,19 @@ class Script(scripts.Script):
 
                 for val in valslist:
                     m = re_range_float.fullmatch(val)
+                    mc = re_range_count_float.fullmatch(val)
                     if m is not None:
                         start = float(m.group(1))
                         end = float(m.group(2))
                         step = float(m.group(3)) if m.group(3) is not None else 1
 
                         valslist_ext += np.arange(start, end + step, step).tolist()
+                    elif mc is not None:
+                        start = float(mc.group(1))
+                        end   = float(mc.group(2))
+                        num   = int(mc.group(3)) if mc.group(3) is not None else 1
+                        
+                        valslist_ext += np.linspace(start = start, stop = end, num = num).tolist()
                     else:
                         valslist_ext.append(val)
 
@@ -184,14 +215,19 @@ class Script(scripts.Script):
             return process_images(pc)
 
         processed = draw_xy_grid(
+            p,
             xs=xs,
             ys=ys,
-            x_label=lambda x: x_opt.format_value(p, x_opt, x),
-            y_label=lambda y: y_opt.format_value(p, y_opt, y),
-            cell=cell
+            x_labels=[x_opt.format_value(p, x_opt, x) for x in xs],
+            y_labels=[y_opt.format_value(p, y_opt, y) for y in ys],
+            cell=cell,
+            draw_legend=draw_legend
         )
 
         if opts.grid_save:
-            images.save_image(processed.images[0], p.outpath_grids, "xy_grid", prompt=p.prompt, seed=processed.seed, p=p)
+            images.save_image(processed.images[0], p.outpath_grids, "xy_grid", prompt=p.prompt, seed=processed.seed, grid=True, p=p)
+
+        # restore checkpoint in case it was changed by axes
+        modules.sd_models.reload_model_weights(shared.sd_model)
 
         return processed

style.css

@@ -1,38 +1,106 @@
 .output-html p {margin: 0 0.5em;}
-.performance { font-size: 0.85em; color: #444; }
 
-#txt2img_generate, #img2img_generate{
-    max-width: 13em;
+.performance {
+    font-size: 0.85em;
+    color: #444;
+    display: flex;
+    justify-content: space-between;
+    white-space: nowrap;
 }
 
-#img2img_interrogate{
-    max-width: 10em;
+.performance .time {
+    margin-right: 0;
+}
+
+.performance .vram {
+    margin-left: 0;
+    text-align: right;
+}
+
+#generate{
+    min-height: 4.5em;
+}
+
+@media screen and (min-width: 2500px) {
+    #txt2img_gallery, #img2img_gallery {
+        min-height: 768px;
+    }
+}
+
+#txt2img_gallery img, #img2img_gallery img{
+    object-fit: scale-down;
+}
+
+.justify-center.overflow-x-scroll {
+    justify-content: left;
+}
+
+.justify-center.overflow-x-scroll button:first-of-type {
+    margin-left: auto;
+}
+
+.justify-center.overflow-x-scroll button:last-of-type {
+    margin-right: auto;
+}
+
+#random_seed, #random_subseed, #reuse_seed, #reuse_subseed{
+    min-width: auto;
+    flex-grow: 0;
+    padding-left: 0.25em;
+    padding-right: 0.25em;
+}
+
+#seed_row, #subseed_row{
+    gap: 0.5rem;
+}
+
+#subseed_show_box{
+    min-width: auto;
+    flex-grow: 0;
+}
+
+#subseed_show_box > div{
+    border: 0;
+    height: 100%;
 }
 
 #subseed_show{
-    min-width: 6em;
-    max-width: 6em;
+    min-width: auto;
+    flex-grow: 0;
+    padding: 0;
 }
 
 #subseed_show label{
     height: 100%;
 }
 
-#txt2img_roll{
+#roll{
     min-width: 1em;
     max-width: 4em;
+    margin: 0.5em;
 }
 
-#style_index{
-    min-width: 9em;
-    max-width: 9em;
-    padding-left: 0;
-    padding-right: 0;
+#style_apply, #style_create, #interrogate{
+    margin: 0.75em 0.25em 0.25em 0.25em;
+    min-width: 3em;
+}
+
+#style_pos_col, #style_neg_col{
+    min-width: 4em !important;
+}
+
+#style_index, #style2_index{
+    margin-top: 1em;
+}
+
+.gr-form{
+    background: transparent;
 }
 
 #toprow div{
     border: none;
     gap: 0;
+    background: transparent;
 }
 
 #resize_mode{
@@ -43,10 +111,10 @@ button{
     align-self: stretch !important;
 }
 
-#img2img_prompt, #txt2img_prompt, #img2img_negative_prompt, #txt2img_negative_prompt{
+#prompt, #negative_prompt{
     border: none !important;
 }
-#img2img_prompt textarea, #txt2img_prompt textarea, #img2img_negative_prompt textarea, #txt2img_negative_prompt textarea{
+#prompt textarea, #negative_prompt textarea{
     border: none !important;
 }
 
@@ -134,8 +202,12 @@ input[type="range"]{
 }
 
 #txt2img_negative_prompt, #img2img_negative_prompt{
-    flex: 0.3;
-    min-width: 10em;
+}
+
+#progressbar{
+  position: absolute;
+  z-index: 1000;
+  right: 0;
 }
 
 .progressDiv{
@@ -161,3 +233,66 @@ input[type="range"]{
   border-radius: 8px;
 }
 
+#lightboxModal{
+  display: none;
+  position: fixed;
+  z-index: 900;
+  padding-top: 100px;
+  left: 0;
+  top: 0;
+  width: 100%;
+  height: 100%;
+  overflow: auto;
+  background-color: rgba(20, 20, 20, 0.95);
+}
+
+.modalClose {
+  color: white;
+  position: absolute;
+  top: 10px;
+  right: 25px;
+  font-size: 35px;
+  font-weight: bold;
+}
+
+.modalClose:hover,
+.modalClose:focus {
+  color: #999;
+  text-decoration: none;
+  cursor: pointer;
+}
+
+#modalImage {
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+    margin-top: auto;
+    width: auto;
+}
+
+.modalPrev,
+.modalNext {
+  cursor: pointer;
+  position: absolute;
+  top: 50%;
+  width: auto;
+  padding: 16px;
+  margin-top: -50px;
+  color: white;
+  font-weight: bold;
+  font-size: 20px;
+  transition: 0.6s ease;
+  border-radius: 0 3px 3px 0;
+  user-select: none;
+  -webkit-user-select: none;
+}
+
+.modalNext {
+  right: 0;
+  border-radius: 3px 0 0 3px;
+}
+
+.modalPrev:hover,
+.modalNext:hover {
+  background-color: rgba(0, 0, 0, 0.8);
+}

webui-user.sh

@@ -0,0 +1,40 @@
+#!/bin/bash
+###########################################
+# Change the variables below to your need:#
+###########################################
+
+# Install directory without trailing slash
+install_dir="/home/$(whoami)"
+
+# Name of the subdirectory (defaults to stable-diffusion-webui)
+clone_dir="stable-diffusion-webui"
+
+# Commandline arguments for webui.py, for example: export COMMANDLINE_ARGS=(--medvram --opt-split-attention)
+export COMMANDLINE_ARGS=""
+
+# python3 executable
+python_cmd="python3"
+
+# git executable
+#export GIT=""
+
+# python3 venv without trailing slash (defaults to ${install_dir}/${clone_dir}/venv)
+venv_dir="venv"
+
+# install command for torch
+export TORCH_COMMAND="pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113"
+
+# Requirements file to use for stable-diffusion-webui
+#export REQS_FILE=""
+
+# Fixed git repos
+#export K_DIFFUSION_PACKAGE=""
+#export GFPGAN_PACKAGE=""
+
+# Fixed git commits
+#export STABLE_DIFFUSION_COMMIT_HASH=""
+#export TAMING_TRANSFORMERS_COMMIT_HASH=""
+#export CODEFORMER_COMMIT_HASH=""
+#export BLIP_COMMIT_HASH=""
+
+###########################################

webui.py

@@ -3,13 +3,8 @@ import threading
 
 from modules.paths import script_path
 
-import torch
-from omegaconf import OmegaConf
-
 import signal
 
-from ldm.util import instantiate_from_config
-
 from modules.shared import opts, cmd_opts, state
 import modules.shared as shared
 import modules.ui
@@ -24,6 +19,7 @@ import modules.extras
 import modules.lowvram
 import modules.txt2img
 import modules.img2img
+import modules.sd_models
 
 
 modules.codeformer_model.setup_codeformer()
@@ -33,31 +29,19 @@ shared.face_restorers.append(modules.face_restoration.FaceRestoration())
 esrgan.load_models(cmd_opts.esrgan_models_path)
 realesrgan.setup_realesrgan()
 
-
-def load_model_from_config(config, ckpt, verbose=False):
-    print(f"Loading model [{shared.sd_model_hash}] from {ckpt}")
-    pl_sd = torch.load(ckpt, map_location="cpu")
-    if "global_step" in pl_sd:
-        print(f"Global Step: {pl_sd['global_step']}")
-    sd = pl_sd["state_dict"]
-
-    model = instantiate_from_config(config.model)
-    m, u = model.load_state_dict(sd, strict=False)
-    if len(m) > 0 and verbose:
-        print("missing keys:")
-        print(m)
-    if len(u) > 0 and verbose:
-        print("unexpected keys:")
-        print(u)
-    if cmd_opts.opt_channelslast:
-        model = model.to(memory_format=torch.channels_last)
-    model.eval()
-    return model
-
-
 queue_lock = threading.Lock()
 
 
+def wrap_queued_call(func):
+    def f(*args, **kwargs):
+        with queue_lock:
+            res = func(*args, **kwargs)
+
+        return res
+
+    return f
+
+
 def wrap_gradio_gpu_call(func):
     def f(*args, **kwargs):
         shared.state.sampling_step = 0
@@ -80,33 +64,8 @@ def wrap_gradio_gpu_call(func):
 
 modules.scripts.load_scripts(os.path.join(script_path, "scripts"))
 
-try:
-    # this silences the annoying "Some weights of the model checkpoint were not used when initializing..." message at start.
-
-    from transformers import logging
-
-    logging.set_verbosity_error()
-except Exception:
-    pass
-
-with open(cmd_opts.ckpt, "rb") as file:
-    import hashlib
-    m = hashlib.sha256()
-
-    file.seek(0x100000)
-    m.update(file.read(0x10000))
-    shared.sd_model_hash = m.hexdigest()[0:8]
-
-sd_config = OmegaConf.load(cmd_opts.config)
-shared.sd_model = load_model_from_config(sd_config, cmd_opts.ckpt)
-shared.sd_model = (shared.sd_model if cmd_opts.no_half else shared.sd_model.half())
-
-if cmd_opts.lowvram or cmd_opts.medvram:
-    modules.lowvram.setup_for_low_vram(shared.sd_model, cmd_opts.medvram)
-else:
-    shared.sd_model = shared.sd_model.to(shared.device)
-
-modules.sd_hijack.model_hijack.hijack(shared.sd_model)
+shared.sd_model = modules.sd_models.load_model()
+shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights(shared.sd_model)))
 
 
 def webui():
@@ -130,6 +89,7 @@ def webui():
         server_port=cmd_opts.port,
         debug=cmd_opts.gradio_debug,
         auth=[tuple(cred.split(':')) for cred in cmd_opts.gradio_auth.strip('"').split(',')] if cmd_opts.gradio_auth else None,
+        inbrowser=cmd_opts.autolaunch,
     )
 
 

webui.sh

@@ -0,0 +1,139 @@
+#!/bin/bash
+#################################################
+# Please do not make any changes to this file,  #
+# change the variables in webui-user.sh instead #
+#################################################
+# Read variables from webui-user.sh
+# shellcheck source=/dev/null
+if [[ -f webui-user.sh ]]
+then
+    source ./webui-user.sh
+fi
+
+# Set defaults
+# Install directory without trailing slash
+if [[ -z "${install_dir}" ]]
+then
+    install_dir="/home/$(whoami)"
+fi
+
+# Name of the subdirectory (defaults to stable-diffusion-webui)
+if [[ -z "${clone_dir}" ]]
+then
+    clone_dir="stable-diffusion-webui"
+fi
+
+# python3 executable
+if [[ -z "${python_cmd}" ]]
+then
+    python_cmd="python3"
+fi
+
+# git executable
+if [[ -z "${GIT}" ]]
+then
+    export GIT="git"
+fi
+
+# python3 venv without trailing slash (defaults to ${install_dir}/${clone_dir}/venv)
+if [[ -z "${venv_dir}" ]]
+then
+    venv_dir="venv"
+fi
+
+# install command for torch
+if [[ -z "${TORCH_COMMAND}" ]]
+then
+    export TORCH_COMMAND="pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113"
+fi
+
+# Do not reinstall existing pip packages on Debian/Ubuntu
+export PIP_IGNORE_INSTALLED=0
+
+# Pretty print
+delimiter="################################################################"
+
+printf "\n%s\n" "${delimiter}"
+printf "\e[1m\e[32mInstall script for stable-diffusion + Web UI\n"
+printf "\e[1m\e[34mTested on Debian 11 (Bullseye)\e[0m"
+printf "\n%s\n" "${delimiter}"
+
+# Do not run as root
+if [[ $(id -u) -eq 0 ]]
+then
+    printf "\n%s\n" "${delimiter}"
+    printf "\e[1m\e[31mERROR: This script must not be launched as root, aborting...\e[0m"
+    printf "\n%s\n" "${delimiter}"
+    exit 1
+else
+    printf "\n%s\n" "${delimiter}"
+    printf "Running on \e[1m\e[32m%s\e[0m user" "$(whoami)"
+    printf "\n%s\n" "${delimiter}"
+fi
+
+if [[ -d .git ]]
+then
+    printf "\n%s\n" "${delimiter}"
+    printf "Repo already cloned, using it as install directory"
+    printf "\n%s\n" "${delimiter}"
+    install_dir="${PWD}/../"
+    clone_dir="${PWD##*/}"
+fi
+
+# Check prequisites
+for preq in git python3
+do
+    if ! hash "${preq}" &>/dev/null
+    then
+        printf "\n%s\n" "${delimiter}"
+        printf "\e[1m\e[31mERROR: %s is not installed, aborting...\e[0m" "${preq}"
+        printf "\n%s\n" "${delimiter}"
+        exit 1
+    fi
+done
+
+if ! "${python_cmd}" -c "import venv" &>/dev/null
+then
+    printf "\n%s\n" "${delimiter}"
+    printf "\e[1m\e[31mERROR: python3-venv is not installed, aborting...\e[0m"
+    printf "\n%s\n" "${delimiter}"
+    exit 1
+fi
+
+printf "\n%s\n" "${delimiter}"
+printf "Clone or update stable-diffusion-webui"
+printf "\n%s\n" "${delimiter}"
+cd "${install_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/, aborting...\e[0m" "${install_dir}"; exit 1; }
+if [[ -d "${clone_dir}" ]]
+then
+    cd "${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; }
+    "${GIT}" pull
+else
+    "${GIT}" clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git "${clone_dir}"
+    cd "${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; }
+fi
+
+printf "\n%s\n" "${delimiter}"
+printf "Create and activate python venv"
+printf "\n%s\n" "${delimiter}"
+cd "${install_dir}"/"${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; }
+if [[ ! -d "${venv_dir}" ]]
+then
+    "${python_cmd}" -m venv "${venv_dir}"
+    first_launch=1
+fi
+# shellcheck source=/dev/null
+if [[ -f "${venv_dir}"/bin/activate ]]
+then
+    source "${venv_dir}"/bin/activate
+else
+    printf "\n%s\n" "${delimiter}"
+    printf "\e[1m\e[31mERROR: Cannot activate python venv, aborting...\e[0m"
+    printf "\n%s\n" "${delimiter}"
+    exit 1
+fi
+
+printf "\n%s\n" "${delimiter}"
+printf "Launching launch.py..."
+printf "\n%s\n" "${delimiter}"
+"${python_cmd}" launch.py