Please create a node to make Adaptive Guidance to work with Ksampler

[wardensc2]

Hi @blepping

Can you make a node to make Adaptive Guidance to work with Ksampler similar to what you do with Detail Daemon
https://github.com/asagi4/ComfyUI-Adaptive-Guidance

Adaptive Guidance help us to use negative prompt in Flux in the the first few steps and only sacrifice generate speed for a few early steps however it's not working well with Detail Daemon. My plan is make negative prompt work for a few step to create the good image with good hand, feet and style then Detail Daemon for the rest of steps to make image for detail. However it's only work with Ksampler which can separate step period. Can you support us ?

Here is the image of Adaptive Guidance

Thank you in advance

[blepping]

Can you make a node to make Adaptive Guidance to work with Ksampler similar to what you do with Detail Daemon

Hmm, it might be possible to do this as a sampler wrapper that forces a certain guider for a sampler (so you'd still need to use it with the bleh preset stuff). However I'm not sure you actually need this feature.

However it's only work with Ksampler which can separate step period.

What do you mean? It's possible to do start/end steps with a custom sampler also using a SplitSigmas node (built in). For example:

high_sigmas here is the first 10 steps, low_sigmas is the last 10 steps. Perhaps I misunderstood the issue.

[wardensc2]

Can you make a node to make Adaptive Guidance to work with Ksampler similar to what you do with Detail Daemon

Hmm, it might be possible to do this as a sampler wrapper that forces a certain guider for a sampler (so you'd still need to use it with the bleh preset stuff). However I'm not sure you actually need this feature.

However it's only work with Ksampler which can separate step period.

What do you mean? It's possible to do start/end steps with a custom sampler also using a SplitSigmas node (built in). For example:

high_sigmas here is the first 10 steps, low_sigmas is the last 10 steps. Perhaps I misunderstood the issue.

Hi @blepping

Thank for your quick answer. So how do I use high sigma and low sigma.

For example I want Adaptive Guidance work with first 10 steps of high sigma, now I want Detail Daemon work with last 10 steps of low sigma. Can you give me an example workflow for it.

[blepping]

Here's an example of how you could use different guiders/samplers for different steps with SamplerCustomAdvanced:

I don't have AdaptiveGuider installed, PerpNegGuider is the one that is used for the first 10 steps there, so that's the one you would want to swap out, probably.

If that looks complicated, it really just boils down to this setup:

And you are free to use different samplers/guiders for each SamplerCustomAdvanced. Using BlehDisableNoise because because ComfyUI has a bug regarding non-deterministic generations ( comfyanonymous/ComfyUI#4518 ) which apparently will not be fixed in our lifetime despite being known for close to a year, multiple people submitting fixes (including myself) and even comfy himself saying he was going to accept it. Grumble grumble.

[pivtienduc]

Here's an example of how you could use different guiders/samplers for different steps with SamplerCustomAdvanced:

And you are free to use different samplers/guiders for each SamplerCustomAdvanced. Using BlehDisableNoise because because ComfyUI has a bug regarding non-deterministic generations ( comfyanonymous/ComfyUI#4518 ) which apparently will not be fixed in our lifetime despite being known for close to a year, multiple people submitting fixes (including myself) and even comfy himself saying he was going to accept it. Grumble grumble.

Thank you so much. By can you give me an example of how to use your BlehSetSamplerPreset

I try to use load model or latent into anyinput of your node but still ComfyUI still said I haven't define bleh_preset_0 yet

Thank in advance

[blepping]

can you give me an example of how to use your BlehSetSamplerPreset

Sure. The main thing to keep in mind is the BlehSetSamplerPreset node has to run before anything uses that preset. ComfyUI runs nodes based on how their results are required. Just for example, in the little example image above, the first SamplerCustomAdvanced has to run before the second one, because the second one is using its output. Nodes also have to get run again when their inputs change.

Of course, you can't always know the order for sure. As another example, for the first SamplerCustomAdvanced, will the RandomNoise node run first or will the KSamplerSelect node run first? We don't know - the other those nodes run is basically random. Doesn't really matter for your question about setting the preset, but I just thought I'd mention it.

Anyway, to ensure BlehSetSamplerPreset runs first, you can do something like this:

any_input just passes through. We know KSampler needs a latent as input, so it can't run before it runs Empty Latent Image and BlehSetSamplerPreset - and at that point the preset will have been set and is available.

There's one other thing to be aware of if you're using the same preset in multiple places ( see https://github.com/blepping/ComfyUI-bleh#blehsetsamplerpreset ). Note this isn't the case with what you're trying to do right now. Anyway, just for future reference:

dummy_opt doesn't actually get used, but since nodes have to run again if their inputs change, you can use that connection with something like the Primitive integer node to force the node to run again whenever you want. i.e. if you run the workflow, then change the Primitive integer to 2 then ComfyUI will have to run BlehSetSamplerPreset again as well as any downstream nodes that depended on its input (and then anything that depended on those nodes and so on).

Hopefully this makes sense! I have a bad habit of making my explanations too complicated, so if it's not clear please let me know.

[pivtienduc]

can you give me an example of how to use your BlehSetSamplerPreset

Sure. The main thing to keep in mind is the BlehSetSamplerPreset node has to run before anything uses that preset. ComfyUI runs nodes based on how their results are required. Just for example, in the little example image above, the first SamplerCustomAdvanced has to run before the second one, because the second one is using its output. Nodes also have to get run again when their inputs change.

Of course, you can't always know the order for sure. As another example, for the first SamplerCustomAdvanced, will the RandomNoise node run first or will the KSamplerSelect node run first? We don't know - the other those nodes run is basically random. Doesn't really matter for your question about setting the preset, but I just thought I'd mention it.

Anyway, to ensure BlehSetSamplerPreset runs first, you can do something like this:

dummy_opt doesn't actually get used, but since nodes have to run again if their inputs change, you can use that connection with something like the Primitive integer node to force the node to run again whenever you want. i.e. if you run the workflow, then change the Primitive integer to 2 then ComfyUI will have to run BlehSetSamplerPreset again as well as any downstream nodes that depended on its input (and then anything that depended on those nodes and so on).

Hopefully this makes sense! I have a bad habit of making my explanations too complicated, so if it's not clear please let me know.

I understand a bit now. I don't have PC at the moment but as far as I understand: First we need to run workflow process to create a preset first. For example as your workflow show:

1 - We put an empty latent size in to your BlehSetSamplerPreset then connect to a Ksampler to make it run. So if I want to create a 1920x1080 size do I need to change the empty latent size to 1920x1080, and what happen if I use face detailer which empty lantent size will change randomly base on the image face size.

2- After we get the preset should I do something like your guide in Daemon Detailer to make Ksampler work. I wonder if I use unet loader not checkpoint loader, does it work ?

Please explain this to me. Thank you so much

[blepping]

Sorry if I wasn't clear, I wasn't saying that you necessarily would use an Empty Latent node or a KSampler - just an example to show the general structure.

To put it a different way, if you have a workflow set up like Node A -> Node B where you're feeding the output of Node A into Node B then you know Node A has to run first: Node B needs its result, which is only available after it runs.

Using that logic and assuming Node B uses a bleh preset, you can ensure the BlehSetSamplerPreset node runs first by changing that to Node A -> BlehSetSamplerPreset -> Node B. It doesn't matter what types of nodes Node A and Node B are or what you're passing through the BlehSetSamplerPreset node. It doesn't care about the contents of the any_input input - it only acts as a bridge and returns exactly the same thing you connected to it.

Does that make sense?

[pivtienduc]

Sorry if I wasn't clear, I wasn't saying that you necessarily would use an Empty Latent node or a KSampler - just an example to show the general structure.

To put it a different way, if you have a workflow set up like Node A -> Node B where you're feeding the output of Node A into Node B then you know Node A has to run first: Node B needs its result, which is only available after it runs.

Using that logic and assuming Node B uses a bleh preset, you can ensure the BlehSetSamplerPreset node runs first by changing that to Node A -> BlehSetSamplerPreset -> Node B. It doesn't matter what types of nodes Node A and Node B are or what you're passing through the BlehSetSamplerPreset node. It doesn't care about the contents of the any_input input - it only acts as a bridge and returns exactly the same thing you connected to it.

Does that make sense?

Yes I understand but does Node A -> BlehSetSamplerPreset -> Node B have to be output an result to set the preset.
As your reference image show: Empty Latent -> BlehSetSamplerPreset -> Ksampler.

Do I need to connect Ksampler to Vae Decode and Preview Image to make creating preset work.
Empty Latent -> BlehSetSamplerPreset -> Ksampler -> Vae Decode -> Preview Image

And after Preset was set. I just need to use Ksampler and select sample name: Bleh_preset_0 and I don't need to use BlehSetSamplerPreset or Sample Custom Advance again right ?

[blepping]

Yes I understand but does Node A -> BlehSetSamplerPreset -> Node B have to be output an result to set the preset.

If I understand the question, you mean something like if you just have a stray Empty Latent Image node that's not connected to anything and then you make that Empty Latent Image -> BlehSetSamplerPreset (still not connecting the output to anything) will that set the preset? The answer is no: none of those nodes will run in either case because no other nodes depend on their input.

ComfyUI has a distinction between normal nodes and "output" nodes. Output nodes would be something like Save Image, Preview Image - those nodes run because they produce an output even though a Save Image node won't have its output connected to anything (and in fact does not actually have an output at all).

Do I need to connect Ksampler to Vae Decode and Preview Image to make creating preset work.

The simplest approach is just use BlehSetSamplerPreset as a bridge for some input the sampler you're using (doesn't matter whether it's a KSampler, FaceDetailer, SamplerCustomAdvanced, whatever) will require. That way you know the BlehSetSamplerPreset will run before the sampler that uses the preset.

And after Preset was set. I just need to use Ksampler and select sample name: Bleh_preset_0 and I don't need to use BlehSetSamplerPreset or Sample Custom Advance again right ?

Short answer: Correct.

Longer answer: The presets get saved into the running ComfyUI server. So if for example you set up your workflow so the preset got set, then you delete the BlehSetSamplerPreset node and restart the ComfyUI server - at that point the preset doesn't exist anymore. This is not a problem you'll run into if you set up your workflow the way I recommended though.

Also, setting a preset basically has no performance cost and overwriting the preset with the same settings is completely fine.

[pivtienduc]

Yes I understand but does Node A -> BlehSetSamplerPreset -> Node B have to be output an result to set the preset.

Also, setting a preset basically has no performance cost and overwriting the preset with the same settings is completely fine.

Ok I understand now. So basically, I just need to connect BlehSetSamplerPreset to anynode that require an input and run it, since I don't connect an output to the node that connected with BlehSetSamplerPreset node so it won't run so no performance cost but still recognize sample and create a sample name Bleh_preset_0.

After this, I just use normal Ksampler and choose sampler name Bleh_preset_0 then run like normal workflow.

[blepping]

Correct. Just use BlehSetSamplerPreset as a bridge for any input that will be required before the first place that preset actually is used, and after that you're free to use that preset in whatever nodes you want.

I just need to connect BlehSetSamplerPreset to anynode that require an input

The only thing I'd add here is it's not just "any node", but any node that has to run before the place you actually use the preset.

[pivtienduc]

Another thing I want to ask. This is not related to your BlehSetSamplerPreset

It's about dynamic CFG like Adaptive Guidance I talked above.

Due to Flux model is quite heavy if I set CFG >1 the inference speed reduce 50%.

Adaptive Guidance using some formula to run CFG >1 for a few step and calculate the likeliness of the image about 90% to 99% of full run CFG>1 then change to CFG=1 to increase the speed 100%. This method is helping us increase only 30% inference time but get 90% or more of the image with full run CFG>1 which negative prompt worked. And also it does not depend on step, it depend on the % of likeliness we set.

Can split sigma node do the same or we need to set a fixed amout of step run with CFG>1 then another fixed step with CFG=1

[blepping]

Can split sigma node do the same or we need to set a fixed amout of step run with CFG>1 then another fixed step with CFG=1

Sorry, I'm not sure I understand your question. Splitting sigmas just lets you split the steps so you can use a different sampler with different parameters (different guider, different sampler, etc). So I guess the answer would be no?

I'm also not 100% sure I understand the problem you're trying to solve (or if it's exactly the same as the person who opened the issue - I am assuming you're a different person). Maybe something like this?

1. Adaptive Guidance will dynamically decide whether to run just for a couple steps or maybe it will decide to run for all steps.
2. Detail Daemon does not work well when it's enabled at the same time as Adaptive Guidance.
3. You want to avoid the case where they are both running concurrently.

Note that I phrased that as a set of statements but I am just guessing what your problem could be and I also don't know that Adaptive Guidance actually works like that. If that actually is your problem, the conversation so far won't really help you because you'd still want to use the Adaptive Guidance guider for both of them (since it might want to run for the remaining steps as well). If that is the case, then really the only solution would be some kind of integration between Adaptive Guidance and DD (or some kind of feature in Adaptive Guidance that would let it switch samplers depending on whether it is enabled or not).

[pivtienduc]

Can split sigma node do the same or we need to set a fixed amout of step run with CFG>1 then another fixed step with CFG=1

Sorry, I'm not sure I understand your question. Splitting sigmas just lets you split the steps so you can use a different sampler with different parameters (different guider, different sampler, etc). So I guess the answer would be no?

I'm also not 100% sure I understand the problem you're trying to solve (or if it's exactly the same as the person who opened the issue - I am assuming you're a different person). Maybe something like this?

1. Adaptive Guidance will dynamically decide whether to run just for a couple steps or maybe it will decide to run for all steps.
2. Detail Daemon does not work well when it's enabled at the same time as Adaptive Guidance.
3. You want to avoid the case where they are both running concurrently.

Note that I phrased that as a set of statements but I am just guessing what your problem could be and I also don't know that Adaptive Guidance actually works like that. If that actually is your problem, the conversation so far won't really help you because you'd still want to use the Adaptive Guidance guider for both of them (since it might want to run for the remaining steps as well). If that is the case, then really the only solution would be some kind of integration between Adaptive Guidance and DD (or some kind of feature in Adaptive Guidance that would let it switch samplers depending on whether it is enabled or not).

Yeah my question is a bit too much. Now I try to figure out that make Detail Daemon working with Adaptive Guidance together.

Both of them work very well when we don't combine them together.

For example I use Detail Daemon, I get very detail with more content in my generate image however I lost the ability to control mutated limb, hand, feet, plastic skin due to negative prompt is not work with CFG=1.

If I use Adaptive Guidance I get the control ability but lose content in detail.

I try to combine both and get a overnoise image. First I try to make each of them work in a fixed step but now I find out that each of them need to work first few beginning steps to make it's effective. So today I will try to mess with Detail Daemon settings or Adaptive Guidance to make them work well together.

At least I solved the problem face detailer and ksampler does not work with Detail Daemon thank to you

[blepping]

For example I use Detail Daemon, I get very detail with more content in my generate image however I lost the ability to control mutated limb, hand, feet, plastic skin due to negative prompt is not work with CFG=1.

Are you positive using Adaptive Guider is actually doing something when used with Flux? From what I understand, Flux just doesn't use CFG so setting CFG to higher values does nothing except make everything twice as slow.

I looked at the code and it doesn't seem to have any special handling for Flux, so in essence it's just using CFG=1 for part of sampling and CFG=different_value for a different part. Since it's Flux, those both should produce the same result.