greenscreen

a YUV based greenscreen seq plugin for blender 2.43-2.49

Note: Sequence plugins will not work in 2.5
 The plugin api that this was based on will not be ported  to blender 2.5+ 

source: greenscreen1.3.c linux: greenscreen1.3.so windows:greenscreen1.3.dll

many thanks to Ben Cox for the greenscreen1.3 windows .dll

update 3/16/09    source: greenscreen1.4.c    linux:  greenscreen1.4.so 

 

These and compiled binarys for other systems are also available in the Bender plugin repository

  


You might also like to check out my new RGBscreen plugin after you look at this page ,
it has the capability to do transparency and hair:
 http://sites.google.com/site/papsplugins/rgbscreen
There is no doubt in my mind that compositor nodes can probably give you a much better result than anything you can do with a sequence plugin but it takes alot of effort to set up properly. On the other hand, the seq editor seems to process a strip a bit faster than the node editor and hopefully this plugin will give you a decent key relatively easily...

I found a greenscreen video at ribbitfilms.com, It was a free download, 480x320, 30fps, unkeyed...unfortunately it no longer seems to be available. I slowed it down to 10 fps and used it for a quick overview of how the plugin works:

click here to download a much better 8 meg 480x320 version

click here to download a 8 meg mp4 spill/spill+ demo

Recently, there was a Stargate Atlantis greenscreen music video contest on YouTube. I used the footage to give you a practical example of how well the plugin handles keying, spill control, and edge blur:

unfortunately, the YouTube conversion/compression process degraded the quality of the video quite badly ...so here is a link to a much higher quality version at the original resolution (425x318 20 megs)

http://uploader.polorix.net//files/832/greenSGAdemo1k.avi



If you have any questions or problems regarding this plugin, follow this link to the greenscreen tutorial thread
at BlenderArtists.org and post your frustration there.

( BTW: This is also how I code... )

Plugin controls:

  • greenscreen: turns the plugin on or off. If toggled to yellow, it inverts the mask..
  • dv4.1.1: filter to compensate for color info loss due to compression.
  • Ymin & Ymax: minimum and maximum luminance values to key out
  • Uclip & Vclip: blue and red component clip values...basically, the U and V values are how "far" from green they are.
  • display: this chooses what is output from the plugin: 0)normal composited output. 1)Selection aid, 2) alpha mask, 3) Y (luminance), 4) U ( chroma blue), 5) V (chroma red)
  • check / ibuf2: selects the background you want to use when keying. Off is a solid color indicated by the color box below, check is a grey checkerboard pattern, and use ibuf2 (yellow) if a second image is attached to the plugin.
  • edge blur: feathers the edge of the mask with the background
  • blur: the blur factor around the edges... the scale of this blur is 4x what you see in a node blur. Usually the default setting works well...
  • radius: this is the radius of the filter for the blur. Again, the default works well here.
  • info / TMI: print plugin information to the console or (possibly) Too Much Information
  • spill / spill+: gets rid of green spill in the non keyed areas of the image. You can manually adjust the "respill". If toggled to spill+, respill is automatically determined by the chosen background color
  • factor: spill reduction strength
  • Yadj, Uadj, Vadj:use these buttons to adjust the "respill" colors when in spill mode- they are disabled in spill+ mode


getting started

Here are my suggestions for setting up so you can get a quick garbage mask after you use the plugin...feel free to ignore any or all of it ;)

  • Start with a new empty scene. Make sure your render size is the same as your video...The sequencer will automatically resize your clip to the render x,y which will not only make it more difficult to key but it can distort your clip also.
  • In front view add a camera.
  • In top view just put the cursor about 10 bu (blender units) in front of the camera.
  • Push ctrl-numpad0 to make the camera active.
  • Set up your output file path/filename
  • turn off pretty much all render options...osa, shadow, ray, etc will not be needed and just slow things down. This scene will be used for our garbage mask and speed is all we want. Also, osa edges on the mask can cause artifacts in your final output.
  • enable do sequence of course.


Now go to the SE and just add your greenscreen image sequence or movie strip into the SE and then add the plugin. If you want to use another strip for a composite background, add that strip also, select your green strip, shift-select your background strip and then add the plugin. If you can't use your target background, find an image or use a background color that will be similar to your scene. This not only lets you see how it may eventually look, it also comes in handy for the spill control later. Do not use a sequence strip as your background for now because that will slow down the response of the SE to a crawl while you are adjusting the plugin. You can change to a sequence strip later if you want too. The SE also responds much faster to image strips than movies.

plugin settings


<-Lets start with this image.

When you start the plugin you will see something like this->

Now you need to change the settings of the plugin to get a good key. There are several different ways to view your image while you are making the adjustments to aid you. The subject that we have here is quite dark so the black background makes it hard to see what is going on. You can either change the background color or toggle the check button on...

This image gives you a better idea of what is being keyed out->


You can also toggle the check button to yellow...

Then you can see what it looks like with your background img->

Since it's easy to change images in the seq editor, you can even use one image to help you set your keys and then change it for your final output.

<-final composited output - this is the end result after setting key values, use ibuf2, edge blur, and just plain default spill at factor=1.

dv4.1.1

1.4 update: dv filter now will do a horizontal only blur on first click to preserve fields in interlaced video and when clicked to yellow does both horizontal & vertical

 The first thing to do is see if you need the dv4.1.1 filter applied. Start by setting Ymin to 1 so that nothing is keyed out and then zoom in on the preview window and look closely at the colors...look at how "pixelated" the colors are due to 4.1.1 compression. On the right, we can see the Y(luma) channel and below it, the V channel. This shows how the color is at half of the resolution as the luma channel.

In this case, you want the dv4.1.1 filter on. The U and V channels are slightly blurred and give you a much better image to key.


After you are done, just put the Ymin slider back down to somewhere around 0.1 - it doesn't matter exactly where because we will have to adjust it soon anyway.

The selection process:

Here is a different clip and what you will see when you first start the plugin. From the amount of the subject that is keyed out and the green hue of what is left, you can tell that there is going to be alot of green spill. You can also see that you are going to have to use a garbage matte

so we can just concentrate on the area immediately around our subject. This lets you get a tighter key because parts of the background that are not evenly lit are sometimes hard to key out without loosing parts of your subject too.You can worry about the spill and garbage mask later, first you need to adjust the Ymin, Ymax, Uclip, and Vclip values.

The plugin works by keying out any pixel that does ALL of the following:

  • has a lum value between Ymin & Ymax - this helps to keep very dark colors and very light colors
  • has a U value below the Uclip vlaue - this preserves blue-green colors
  • has a V value below the Vclip value - this preserves red -green colors

This is where display mode 1, selection mode, comes in handy. In this mode you will see which pixels are not keyed and which of the channels are stopping it from being keyed out.

  • If the luma value of the pixel is <Ymin or >Ymax, it adds a green component to the display.
  • if pixel U value > Uclip, it adds a blue component to the display.
  • if pixel V value > Vclip, it adds a red component to the display.

<-This image indicates that the Vclip setting is pretty much the only effective setting so far. This image -> shows what you will see after doing some adjustments to the other settings. Note how the colors mix to indicate that more than 1 channel is active ( white means Ymin & U &V, dark purple means U & V, etc)

Getting from the left display to the right is easier than it looks... First of all, if you click on the left image above, you will notice that "pixelated" look again- I haven't applied the dv4.1.1 filter yet. Everything from here on uses the filter.

The easiest way to start is just to adjust each of the sliders until you start to get parts of the background to show up in your display and then back it off a bit.

click on any of the following images for a full size view ( clipped 2x zoom of origional)

Ymin - there is alot of very dark colors here so we can bump up the Ymin value quite a bit to save them. The image on the right has a Ymin =0.304. Note the dark green showing what is "protected" by Ymin.


Ymax - some of the green floor is pretty bright but we can still bring the setting down to preserve some of the lighter areas of the subject as shown in light green. Here we ended up with Ymax=0.834.

Uclip - As you lower this value you wil protect more of the blue-green colors in your image. Uclip=0.391


Vclip - As you lower this value you will protect more of the red-green colors in your image. Vclip=0.421




While adjusting your key. it is often helpful to switch back and forth between the normal mask, inverted mask, display 1,

and the 3 background options [solid color, check, & ibuf2] so you can see how it affects your key.


This is a pretty good key but there seems to be a few spots that could use some tightening up. For instance, below the back knee and a few areas around both shoes. If you go back to display 1 you can see that the problem areas below the knee and behind the front shoe are red.

This means we need a higher Vclip value. Vclip= 0.441 seems to improve that. The bad spot to the right of the back shoe is blue so we need to raise the Uclip value to get rid of it - Uclip= 0.411 seems to be better.



Edge blur is easy. Just toggle it on...

1.4 update : edge blur now will do a horizontal only blur on first click to preserve fields in interlaced video and when clicked to yellow does both horizontal & vertical blur

This is just a simple blur at the edges of your mask to get rid of the jagged edges and help your subject to blend in better. You can play with the blur factor and radius of the of the blur, but in most cases the default values work well. It also works well to help cover the occasional pixel inside your subject that accidentally gets keyed out...  The images on the right are display 2, the alpha mask.  You can render out this view if you want a mask for other compositing later...

 (Remember: toggling the greenscreen button to yellow will invert the mask)   

spill & spill+ control:

1.4 update :the spill factor algorithm is based on the ratios of the three (RGB) color channels.  The factor value has been changed to a default value of  -0.35 and settings between -0.5 and 0.0 are recommended.  If your edges still seem a bit green, lower it and if you find that spill control is adversely affecting certain colors  (like yellow hues are turning orange) raise the factor value.

Spill control is just as easy. Just toggle spill on and raise the factor value until the green is gone -usually you can just set the factor to 1. Spill control tends to darken parts of the image as it removes the green but the plugin preserves the original luma value of each pixel also.

Usually, spill removal stops there but now you can "respill" your image in 2 different ways to better match the lighting of your new background.

spill: If you would like your image to match the new background better, you can now use the Yadj, Uadj, and Vadj levels to change the brightness or color of the spill area . Change the Yadj to affect the overall brightness of the respill. Uadj & Vadj change the blue & red components (YUV colorspace is a bit wierd...you may not get the results you want at first but just play with the controls a bit and you will get the hang of it ). It can be helpful to invert the mask so you can see what spill is doing to the keyed out portions of your clip and and then make adjustments until they are close to the color of your new background

spill+: there is also a second toggle ( to yellow ) which will automatically use the background color as the respill target. The Yadj, Uadj, and Vadj controls are disabled in this mode... all you have to do is change the background color to adjust your respill.

So now we have a pretty good key. Lets see how the whole frame looks...

heh...I told you we were going to need a garbage matte...and I bet it's larger than you thought it would be. That's because to preserve some important dark edges in his shoes and back, we had to actually save some colors that are present in the darker areas of the background.

The garbage mask

I like to do my garbage mask last, because then you know exactly what you need to mask out and how quick and dirty you can afford to be. Some well lit scenes like the the example for the dv filter above actually need no garbage mask at all.

Here is another shot we already have set up. You can see with the the check background and the display 1 view that we have to get rid of the bars at the top and bottom, both right and left edges, and a problem area in the bottom right corner. Since the plugin works pretty quickly, you can even scrub up and down the timeline to see just how creative you are going to have to be with your mask.


Now we need to start setting up our mask. How you choose to make your mask depends on you and how much you have to animate it... curves, shape keys, or just static or moving planes will all work equally well as masks. In this case, I see that all that I need is a static mask and just a rectangular plane with a hole in it for our subject.

I added a plane and extrude-scaled it so it covered camera view, and added a new material to it. Start with red=0, blue=0, green=0.8, and turn off everything but shadeless...this way the mask is evenly colored & the scene doesn't even need a light to render it. Then I deleted the center face and sized the hole so that I think it will allow us to see our subject.


Now we have to add this to the SE. Just to make things easier to keep track of later, move the plugin to channel 5, your background strip to 4( if you have one), and our green strip to 2. (Actually, I knew this was coming so I put them in those channels in the first place...)

  • Now add the garbage mask scene to the SE and put it in channel 1.
  • Select your green strip, shift-select the scene and add alpha over
  • Select the alpha over strip, shift-select your background strip, shift-select the plugin and hit "r"

*Notice I also rendered the original .mov to pngs to speed up the SE because of ffmpeg seek and decode times while I was doing the adjustments to the plugin...


What we have done here is to put our mask over our movie before we key it and thus cover the bad spots with an easily keyed color. Now you can scrub (slowly) through your clip and adjust the mask, add more mask objects and do any simple animating of your mask(s) until it's just the way you want it. Like I said earlier, adding a scene strip to the SE can really slow down the response time and that is why we did everything before to speed up the mask render times.

NOTE: If you change the mask in any way, you will have to exit edit mode & hit the refresh button to have blender rerender that frame so you can see your changes

masking tips:

If you do happen to be lucky enough to have a static mask, it's much faster to just render one image of it and use that image in the SE instead of a scene strip.

On the other hand, if you have alot of animation to do to your mask, you may just want to:

  • render out a working clip with any background that helps you see what needs to be masked...display 1 works well too.
  • put that clip as a background in your 3d camera view while you work on your mask(s) in the animation screen...it will be much faster to work here than in the SE with a scene strip.
  • then you can either add the scene to the SE or render out a clip of the mask and add that after you are done with it.

Obviously, I didn't spend any time on my background and I was lucky enough to have a simple, non-animated mask, but using this method should get you a good result relatively quickly.

Here is a video clip of the results.

final output with spill+ : hardwater20113_0431.mp4

Final note: If you are having trouble getting a good key from your footage, there are a series of excellent videos on YouTube by Mark Apsolon about setting up your greenscreen and lighting your shots. If you follow his advice, your shots will be easy to key...I would suggest you start with How to make awesome green screen (Chroma key)!! part 1

and here is another good lighting video:

Walter Graff teaches greenscreen lighting

greenscreen images courtesy of RogerWicks and CoreyAvitar