Auto Convergence

[JackSkellington90]

Let me start by saying how much I appreciate IW3, it’s completely changed how I watch films. My only wish is that like owl3D I wish Iw3 had an auto convergence option, i spend too much time trying to find the right convergence and after rendering I notice that it’s not ideal for the whole film. Like someone mentioned earlier auto 3D strength would also be useful but I think getting the convergence rate right for me is always the most difficult. That being said I can’t thank the developer enough for iw3 software.

[nagadomi]

Technically, 0.5 is the setting where the pixel shift amount becomes minimal.
I understand that the feature you are requesting is one that adjusts the scene depth — moving the image forward or backward to make it look better for each scene.
However, I consider this not as something with an objectively correct behavior, but rather as a kind of subjective, artistic effect.

If there is some universal and robust method, it could be added as an option, but personally I prefer not to add effects that the original artist did not intend.
Well, 3D conversion itself is already an additional effect, but I see it as a modification on the player side rather than on the content itself.

[3gMan]

The most commonly used and regarded method of convergence is converging on the subject of focus. James Cameron uses it. Avatar and most or all Marvel movies are good examples. Owl3d has a "clear subject" that does this and gets it right about 90-95% of the time. It is also possible to manually change the convergence of each scene in something like Resolve which I do on some conversions, but it takes many hours like 10-100 to get it just right.

There would need to be a way to detect the subject of each frame. The simplest way would be using the sharpest and most in-focus area and converge on that. I am pretty sure there are other variables that should be used like movement and maybe the average brightness of the depthmap. I think Ultralytics YOLO is one such object tracker that could maybe be used or give an idea of how to accomplish this.

Here is an example of a clip where I converged on the subject (usually the eye or face of the person talking etc) in almost every shot manually. Even though it's pretty strong depth, it is also quite comfortable because your eyes are usually looking at the subject of focus which when converged like this, is at screen depth where the two images naturally overlap/converge - instead of your eyes constantly working to focus behind or beyond the screen.

Another explanation from AI

[JackSkellington90]

Thank you for the insight, for most 3D conversions we have iw3 and owl3D, despite the learning curve if I had a choice I would go with iw3 anyday but really the only thing that attracts me to owl3D is the auto convergence feature, as what the other poster said it is quite time consuming to have to change the convergence in each scene, and I have no technical expertise whatsoever, as a matter of fact it took me about a couple of weeks just to do proper conversions, not that I’m complaining, lol. But for my expertise I normally just stick to older films since the type of filming is usually quite similar throughout.

In any case I wanted to let you know how appreciative I am of your work, and I’m sure many here are grateful.

[nagadomi]

@3gMan

Thank you for the detailed explanation.
If the goal is to align zero parallax with the focus or the main object, then the task is quite clear.
As you mentioned, assuming there is depth of field, the focus area could be selected either by detecting the sharp region (without blur) or by using machine learning methods such as object detection or saliency maps.

Focusing on blur might handle camera focus more appropriately, but for non-film content, there may be no depth of field, or motion blur from moving subjects, which limits the cases where this method can be used.
Also, identifying blurred and sharp regions requires some resolution, so even though the method is simple, it may still be computationally heavy.

As for machine learning methods, there are tasks like salient object detection that directly fit this purpose.
You may know that the older version of iw3 had a background removal feature — it’s essentially the same idea.
The result doesn’t need to be high-resolution, and it doesn’t have to run on every frame, so the additional cost shouldn’t be high.
However, since many modern models focus on producing high-resolution results, there might not be a model that balances accuracy and speed well.
Low resolution may also make it hard to identify blur from sharp areas, leading the model to focus on regions different from the camera’s actual focus.

Anyway, to summarize — I can’t point to the best method right now, but adding such a feature is definitely realistic.

[3gMan]

Thanks nagadomi, that would be an amazing advancement for iw3! Another consideration is how fast or slow to change convergence. Scene changes can likely occur instantaneously, but possibly mid-scene convergence changes should be more gradual.

[nagadomi]

Yes, that is the same as updating the depth range in the flicker reduction option.

[JackSkellington90]

I’d be really happy to see these in the upcoming updates, thank you :)

[AIVFI]

Auto Convergence is much needed and I join the request.

I can also suggest the use of the face segmentation model that achieves the best results in eyes segmentation, namely SegFace:
https://arxiv.org/abs/2412.08647
https://github.com/Kartik-3004/SegFace

However, where there are no human eyes there will of course be a need for a video salient object detection model, where I would like to highlight TransFlow:
https://arxiv.org/abs/2507.19789
https://github.com/suhwan-cho/TransFlow

For RGB-D video salient object detection, SMFNet can be used:
https://arxiv.org/abs/2507.21857
https://github.com/Jia-hao999/SMFNet

[JackSkellington90]

Nagadomi, can we expect auto convergence in any of the upcoming updates ? I’m curious if we might get it soon with the new update or if it’s a feature that requires a long process to experiment with.

Thank you, I appreciate everything you do.

[nagadomi]

I've been away for about a month now, and since I only have my phone, I haven't been able to update anything. I'll probably do it sometime next month.
I don't think I'll spend much time on this, but since many other tasks have piled up over the past month, it might take some time before I can start working on it.

[JackSkellington90]

Thank you, I honestly can’t wait to have this feature, it’ll save me a lot of time trying to figure out the convergence setting as a new comer here. I appreciate your effort and everything you’ve provided us here.

[JackSkellington90]

Hello Nagadomi, I've finally found a great way to figure out the right 3D strength for a film. Basically what I do is I do a set of different strengths of the same scene, and use potplayer to view the film in 3D with the red and blue cyan colors. Then I label each with the 3D strength. After that I use Chat GPT AI to recommend the highest 3D strength I can choose while still keeping the film comfortable to view entirely on a large projector, usually it's very accurate in what it recommends so I use that as a starting point.

I know auto convergence would be a huge feature that a lot of people would appreciate, but perhaps we could also one day get a separate software which can look at a specific clip of a video and recommend the best 3D strength or convergence in average.

I understand that preference in 3D strength is subjective but it could really help us get a good starting point to a comfortable 3D strength and adjust it towards our preferences from there. :)

[QuickscopingFTW]

Is this feature being looked at or are there other things taking priority at the moment? either way good job with the da3 implementation, works great now

[nagadomi]

For now, this task is scheduled to be done after two more tasks. However, if a quick task comes up, I may do that one first.