Video-Based Tutorial: Page (1) of 1 - 08/29/11 Email this story to a friend. email article Print this page (Article printing at MyDmn.com).print page facebook

Understanding Video Compositing

Understand how composite modes work in Sony Vegas Pro By Gary Rebholz

Vegas Pro software features powerful video compositing tools. A video composite occurs when you do something to your video that combines two or more streams of video so that they appear simultaneously as a new single image. In Vegas Pro software you can accomplish this task with many different tools, the most obvious being the track motion controls (think about creating a picture-in-picture sequence), certain video filters (like the Cookie Cutter, the Chroma Keyer, and others which enable compositing by creating transparency in the video clips they've been applied to), pan/crop controls, and more.

But one set of compositing controls remain largely unexplored and underutilized by perhaps most Vegas Pro editors. The composite mode options give you a wide range of control over the look of your project, but many people don't understand how they work. So in this article, we'll take a look at some examples that will help you understand how composite modes work and how you can use them to enhance your projects.

Start a new project. Click the Media Generators tab, select Solid Color from the list, and add the Red preset to the beginning of a new track. In the Video Media Generators window, note that the color is represented with the RGB color model. In this model, Red, Green, and Blue are mixed together to create all other colors. The resulting color also has an Alpha value. We'll talk more about alpha values shortly.

This red has an RGB color recipe of R: 255, G: 0, B: 0 as well as an Alpha value of 255. That information will come in handy as we make our way through this discussion. Next, drag the Blue thumbnail onto track 2 on your timeline directly under the red event. In the Video Media Generators window note that this blue has an RGB recipe of R: 0, G: 0, B: 255 with an Alpha value of 255. In the Video Preview window, you don't see the blue at all.

 

These results highlight a critical point in understanding compositing in Vegas Pro software: The video signal flows from the bottom up. Think of this for a moment as sheets of paper. When you put the blue sheet of paper down on your desk, you can see it clearly. That is, until you lay the red sheet of paper over the top of it. Then you can see only the red.

It's the same idea here. The red event on the top track obscures your view of the blue event on the bottom track. But you can use compositing to see a combination of both colors on the screen at the same time.

Vegas Pro gives you many ways to create a video composite. I mentioned a few of these earlier. The technique you choose for creating your composite determines how the two video streams are mixed for final output. For instance, you can lower the Level value of the top track to enable the lower track to show through. Drag the Level slider for Track 1 slowly to the left. As you do, you see more and more of the blue from the event on track 2 show through and blend with the red on the top track. The Video Preview window shows the results of this blend as it passes from red toward purple and on to blue. A Level value of 0% makes the top track completely transparent and you see the completely blue results. Raise the Level slider back to 100%.

Click PLAY or press spacebar to start or stop video

Even with the Level slider on track 1 all the way up, you can use composite modes to create video composites. Vegas Pro uses Source Alpha as the default composite mode for new tracks which means that the application looks at the top video track for transparency. If it finds transparency, it lets the color from track 2 show through. This brings up another critical point to understanding compositing modes: the top track's composite mode dictates how the video information from the video track below it affects your final output. If there is also transparency on track 2, Vegas looks through to track 3 if it exists, and on and on for as many tracks as you have in your project. If there's transparency on the bottom track, Vegas Pro looks through to the black project background. Thus you can use the background as a compositing element as you'll see a bit later in this discussion. In this case, since the red event contains no transparent areas, you don't see the blue event in your Video Preview window.

It's not difficult to understand how the Source Alpha mode works. In this mode, Vegas Pro utilizes the video alpha channel that I mentioned earlier. Essentially, the alpha channel controls color transparency of a video clip. Alpha values range from a minimum of 0 (completely transparent) up to 255 (completely opaque). We saw earlier that the red event has an Alpha value of 255 and that explains why you don't see the blue.

If the video on track 1 contains an alpha value of less than the maximum value, you'll see the video on track 2 show through those transparent areas. Think back to those sheets of paper sitting one on top of the other. Unless the red sheet has one or more holes cut into it (which would represent an alpha value of 0), you won't see the blue paper below. The same thing is true here with our video tracks. But unlike the paper which is completely opaque except where a hole exists to make it completely transparent, your video tracks can have 256 levels of transparency (an Alpha value of anywhere from 0 to 255) so that you can blend colors from one track with the one below it.

Let's see how Vegas Pro software utilizes the alpha channel. Drag the red event to the right so that it no longer sits on top of the blue event. Back in the Media Generators window, select Color Gradient from the list. Look at the Elliptical Transparent to Black preset thumbnail shown in Figure 1. Notice that it contains a checkerboard pattern in the center (as do several other presets). That checkerboard indicates transparency, which you now recognize as an alpha value of something less than 255.


The checkerboard pattern that you see in several of the thumbnails indicates an alpha value of something less than 255: transparency.

Select the Checkerboard generator from the list. Notice that the Vertical Blinds preset thumbnail has cream-colored vertical bars alternating with the same checkerboard pattern you saw a moment ago. Drag this thumbnail to the timeline and position it directly above the blue event. In the Video Preview window you now see a pattern of alternating blue and cream vertical bars. This is because-since the top track uses the Source Alpha compositing mode-the transparent areas between the cream bars on the top event enable the blue from the bottom event to show through.

Now that you understand the most basic composite mode and the concept of Vegas looking at video tracks from the bottom up, let's dive just a bit deeper into a couple of other compositing modes. This will give you more of a sense of how the video signals travel from the bottom track up to create composites. We'll create a few different composites using the Add and Subtract composite modes since those are among the easiest to conceptualize. Simple math will explain the results we see as we experiment. You already noted the RGB recipes for the red and blue events. One more thing worth knowing is that the RGB recipe for white is all values at 255 and the recipe for black is all values at 0.

To get started, delete the vertical blinds event and move the red event back into position over the top of the blue event.

As we've seen, Vegas Pro uses Source Alpha as the default composite mode for any new track. But the application offers many more options. Let's start with the Add composite mode. In this mode, Vegas Pro Adds the colors in the top video track to the colors in the bottom video track. To see this click the Composite Mode button for track 1 and choose Add from the list.

In your Video Preview window you now see the color Magenta as in Figure 2. Let's go to the math to understand exactly what's going on here. On the timeline, the top event is red which has an RGB recipe of 255, 0, 0. The bottom event is blue with an RGB recipe of 0, 0, 255. The Add composite mode simply adds the colors of the top event to those of the bottom event, so that the resulting Red value is 255 from the top event plus 0 from the bottom event which equals 255. The Green component is 0 + 0 = 0, and the Blue component is 0 + 255 = 255. So, we end up with an RGB result of R: 255, G: 0, B: 255. That happens to be the recipe for the color magenta and that's what you see in the Video Preview window.


When you switch track 1 to the Add compositing mode, the red and blue are added to create magenta.

Understanding the Add mode is simple because of the nature of addition. With addition, it doesn't matter whether you add 100 to 150 or 150 to 100; it adds up the same either way. But Subtraction doesn't work that way, so let's discuss what happens when you use Subtract mode. This will help you understand the bottom-up signal flow in Vegas Pro.

Click the Composite Mode button for track 1 and choose Subtract from the list. In the Video Preview window you see what might seem to be somewhat surprising results: Blue. What happened to the red? Again, some simple math explains this result.

Since Vegas Pro compositing works from the bottom up, we're starting again with our solid blue, RGB: 0, 0, 255. From that Vegas Pro software subtracts the values of the red in the top track. So the blue from the bottom event minus the red in the top track with its RGB of 255, 0, 0 yields a final RGB value of 0, 0, 255 which you'll recognize as the recipe for blue. Why? Let's stack it up to make the formula more obvious:

The only potentially surprising result in this subtraction formula lies in the red component math. There we see that 0 - 255 = 0. While in normal math any fourth grader can tell you that this isn't true, in RGB math there can be no value less than 0, so the formula makes perfect sense. (By the way, in the RGB model, the maximum value that a color can have is 255.)

Now, think about what would happen if you did the subtraction the other direction. That is, what if you started with the color on the top track and subtracted the color from the bottom track? You'd end up with much different results. Specifically, you'd end up with red. That's why in order to understand how Vegas Pro compositing works, you need to understand that it works from the bottom up.

Now that you understand this basic compositing signal flow, you can experiment with the other compositing modes and see how they affect your video. The Vegas Pro application help file gives a very good description of the math that Vegas Pro uses in each mode.

Of course, the examples I've used so far aren't very useful in the real world. After all, if you want the color Magenta it'd be a lot easier to add a media generator with the Magenta preset straight away instead of jumping through all of these compositing hoops. But these examples made for very clear math. The math works exactly the same way with any video clips that you composite with the various modes and you can achieve some very interesting results. Figure 3 shows an example of two video clips (shown on the left as they are positioned on my timeline) and the results of compositing them in Add mode on the top right and Subtract mode on the bottom right. You can see how different the compositing results are! Understanding how the composite works is your first step to understanding how you might utilize the various modes to achieve effects like these and others. Remember, with some modes (like Subtract in this example), you'd get a different set of results if the videos were swapped with the other one on top.


Two video clips (originals on left) composited in Add mode (top, right) and Subtract mode (bottom, right).

If we dig deeper yet, more possibilities arise. For instance, you don't have to composite between two tracks. Remember that earlier I mentioned that you can composite the bottom track in your project with the black timeline below it. In many of the composite modes this either has no obvious effect on your video (for instance in Add mode you're adding the top clip to black which results in seeing the top clip) or it turns your video black (for instance, in Subtract mode you're subtracting the top video from black which results in black). But a couple of the composite modes give you always interesting and potentially useful results when you include the black background in your composite.

To see this, add a video clip to the bottom track in your project (make sure there are no clips above it on higher tracks) and set the composite mode for the bottom track to Difference Squared. In Figure 4 you can see the results of my project when I do this. The figure shows the original clip on the left and the composite results on the right. Notice how much richer the colors are in my composite than they were in the original clip.


You can also composite a single clip with the black project background.

Another possibility is that you could add a third track to the mix and create a composite between tracks 2 and 3 in addition to the composite between tracks 1 and 2. What happens? Remember; compositing works from the bottom up. So, first Vegas Pro software takes what's on track 3 and uses the composite mode for track 2 to create a composite between tracks 2 and 3. Then, the resulting composite is composited with track 1 according to the track 1 composite mode setting. Written in a mathematical-like formula (using the Add composite mode) it looks like this: (Track 3 + Track 2) + Track 1 = final result.

I'm no mathematician, but I know enough about it to know that in a mathematical formula you always perform the contents of the parenthesis first. So the formula above in English says, "First add Track 2 to Track 3 to get a result and then add Track 1 to that result to get to your final result."

To see an example of this, place the cursor in your project so that it sits within the color generator events we were using earlier. Change the composite mode of the bottom track back to Source Alpha. Change the composite mode for track 1 to Add so that your Video Preview window shows magenta again. Insert a new video track at the top of your project and drag the Linear Red, Green and Blue preset from the Color Gradient generator so that it sits above the red and blue events. Your Video Preview window no longer shows the magenta since the red/green/ blue blend doesn't contain any transparent areas. Now set the composite mode of the new track 1 to Subtract.

Use your math skills to make sense of the results shown in Figure 5. In the upper-left-hand corner, the blue of the event on track 1 is subtracted from the magenta composite between tracks 2 and 3 and you're left with red. In the lower-right-hand corner, the red of the event on track 1 is subtracted from the magenta composite between tracks 2 and 3 and you're left with blue. The green stripe in the middle of the track 1 event has no effect when subtracted from magenta (since there is no green value in the color magenta) and thus the resulting stripe is magenta.


The composite of track 1 with the results of compositing tracks 2 and 3.

Again, the key to understanding this is the bottom-up video signal flow. First track 3 is composited with track 2 according to the composite mode setting of track 2. Then the resulting composite (the color magenta) is composited with the next track up-track 1-according to the track 1 composite mode setting. And the process works like this for as many tracks as you have in your project.

Now that you understand it, let me introduce a twist of sorts to the bottom-up rule. This twist occurs when you create parent/child relationships between tracks. When you create this type of relationship, you have another compositing tool available to you: Parent Composite Mode. When it comes to compositing, the tracks in a parent/child relationship stand apart from tracks that are outside of that relationship. Vegas Pro software first composites the tracks in any parent/child relationships and then uses the parent composite mode to composite those results with any tracks outside of the relationship. This is true even if the tracks within the parent/child relationship are on top of other tracks in the timeline.

Further, the bottom track of a parent/child set looks past any tracks below it (outside of the parent/child set) to the black project background to create its composite.

Let's take a look. Add two video clips to tracks 1 and 2 and move the red/green/blue color event from track 1 so that it sits on track 3 directly below the new video clips. Set the composite mode for track 1 to Subtract and of track 2 to Add. Figure 6 shows my results.


A composite of two tracks and a color event.

Now, to create a parent/child relationship between tracks 1 and 2, click the Make Compositing Child button for track 2. Figure 7 shows the results of my composite now and they're quite different than the composite result in Figure 6.


Creating a parent/child relationship between tracks 1 and 2 changes the composite results dramatically.

To explain this result, work from the bottom up. But remember, parent/child relationships happen first. And don't forget that the bottom track in the parent/child set looks past any tracks outside of the set, even if they sit below the parent/child tracks as in this example. So, first (since track 2 is set to Add) Vegas Pro software adds the video on track 2 to the black background (skipping over track 3 which is outside of the set) and then subtracts the video on track 1 from the resulting composite. Track 3 is ignored completely.

Change the track 2 composite mode to Difference Squared to deepen the colors as we did earlier. Figure 8 shows my results.


The colors are richer and deeper with track 2 set to Difference Squared composite mode.

Set the track 2 composite mode to other modes to see how it affects your results. I'll set mine to Hard Light. Now, click the Parent Composite Mode button for track 1 and choose different composite modes from the list to see how they affect the video. This brings the color event on the third track into the composite as the result of the composited tracks within the parent/child set are composited with the bottom track. Figure 9 shows my results with my parent composite mode for track 1 set to Hard Light. I don't know how these settings will affect the video clips you've chosen, but you can see that it gives my video a very ethereal, eerie look.


Using different combinations of track and parent composite modes gives you a wide range of results.

So, there's a basic look at how compositing works in Vegas Pro. I know some of the examples we worked through here are strictly theoretical, but I also tried to give you a few practical examples that hopefully spark some ideas that you can use to enhance your projects. Remember to check the application help file for a discussion of the method behind each individual composting mode. Now that you understand the video signal flow, these tools are more accessible to you and you can start working with them to see what you can create. Above all, experiment with these tools! You'll be surprised at what you will come up with.

Remember, if you'd like more training on Vegas Pro and our other software titles, visit the training zone on our website . There you'll find many free tutorial videos and archived versions of all of our recent live webinars in addition to our Seminar Series training DVDs and other resources.

 


Page: 1


Gary Rebholz, is the training manager for Sony Creative Software. Gary produces the popular Seminar Series training packages for Vegas Pro, ACID Pro, and Sound Forge software.
He is also co-author of the book Digital Video and Audio Production. Gary has conducted countless hands-on classes in the Sony Creative Software training center, as well as at tradeshows such as the National Association of Broadcasters show.
Related Keywords:Sony Vegas Pro, Compositing, video compositing, video editing

HOT THREADS on DMN Forums
Content-type: text/html  Rss  Add to Google Reader or
Homepage    Add to My AOL  Add to Excite MIX  Subscribe in
NewsGator Online 
Real-Time - what users are saying - Right Now!

@ Copyright, 2014 Digital Media Online, All Rights Reserved