Before starting this, I would like to credit koitsu for writing a guide/blog on how to install and use SCFH DSF. The guide is a bit generic, and doesn't necessarily cover things as in-depth as some may like, though, which is why I'm writing this.
I would also like to thank Destiny for his Beginner's Guide to Desktop Streaming, which entails how to use VH Screen Capture with FMLE 3.2. I recommend reading it thoroughly, since I focus mainly on using SCFH itself, rather than FMLE - while it is a different version of FME, many of the concepts are applicable (including Virtual Audio Cable, which is included in the guide).
DirectShow Source Filter for Screen Cast (aka SCFH) is a screen capture program like many other virtual webcams out there, including VHScreenCap. However, VHScreenCap and VHMultiCam are very preliminary programs made by Split Media Labs (i.e. the people who made XSplit) that have not been updated in a while, so they lack many of the functionalities of the XSplit capture device itself. SCFH, while unable to encode video by itself, can be used to capture screen regions in conjunction with FME to achieve some of the same abilities.
Essentially, this is an alternative to using XSplit for those unable to purchase the program.
-SCFH DSF - Website - Download
-Flash Media Encoder 2.5 - Website
Note: The reason FME 2.5 is used instead of the latest edition (FMLE 3.2) is that FME 2.5 uses much fewer resources and is less hectic of an interface to use with SCFH. FMLE 3.2 can be used, although it may not produce the same results.
Step 1: Download both SCFH and FME. Place the SCFH folder somewhere you can easily find it (i.e. Program Files), as the installation process will not create a new folder in Program Files, or wherever you want. If you are running a 32-bit system, simply run install.bat; if you are running a 64-bit system, run install64.bat. All this will do is make some registry changes and add an SCFH.exe to the folder. Create a short cut of SCFH.exe and place it wherever. The installation for FME 2.5 is self-explanatory.
Step 2: Open up FME.exe (Flash Media Encoder under Adobe in the Start Menu), but do not do anything with it yet. Now, open up SCFH.exe (or use the shortcut you created). It's time to configure SCFH.
This is the process selection window of SCFH. Basically, this is what tells SCFH which process/application it should work for (i.e. if you have FME up while using Windows Messenger or Skype). Hit Refresh to let it search for processes again (which includes anything opened after opening up SCFH). After selecting FlashMediaEncoder.exe, hit OK. The Force Windows Aero checkbox will allow you to keep Aero enabled while streaming; by default, it will switch you to Basic. I recommend keeping it unchecked, because you can take a hit in streaming performance if Aero is enabled.
The window that pops up should look like this. The Tools option lets you convert to screen coordinates (i.e. desktop) or convert to application coordinates. Changing to application coordinates means the screen capture will record only the application, ignoring objects that are on top or anything outside the application that would otherwise be shown with regulard desktop recording, such as the top bar and the taskbar.
Languages only contains English and "External", which will screw everything up. Try it out, if you want.
The bar right below the top bar is the multi-region handler, which lets you switch between multiple screen regions to edit (which will be explained later).
Skipping down to the Options, there are a bunch of check boxes and drop boxes you can change:
-Thread Num is the number of threads you want SCFH to use. If you are using a quad core and are streaming a double-threaded game such as SC2, you would want to select 2 (and set the specific processors to use in Task Manager).
-Resize Method lets you choose between Bilinear or Nearest. I find better performance and quality with Bilinear, although you can experiment to see which works better for you.
-Show Mouse Cursor can be checked or unchecked, depending on whether or not you want viewers to be able to see your cursor.
-Show Layered Window results in a performance hit for nothing noticeable, so keep it unchecked.
-Keep Aspect Ratio means and screen recording you do will maintain its ratio even when outputted at a different ratio - keep this checked, otherwise you'll have stretching occur.
-Enable Enlargement is to enable stretching of the capture area when the resoluion of the capture is smaller than the resolution of the output. Keep it unchecked, because it ruins the point of Aspec Ratio.
-As for Oversampling, it really depends on how well your hardware can handle it (think anti-aliasing). Keep it unchecked.
Step 3: There are multiple ways to record the screen with SCFH. Returning to the SCFH UI, hitting Area Selection will result in a green box appearing on your screen, which is moveable and resizeable.
You can see how the box looks like in the image above. To apply the green box, double-click on it. You will see that the coordinates and size of the box are automatically applied to the X/Y and Size boxes above Area Selection.
You can also manually input the coordinates and size of the area you want to record by simply typing it into the boxes.
X is the number of pixels away from the left (i.e. 100 would mean the recording starts at 100 pixels from the left/toward the center), while Y is the number of pixels away from the top (i.e. 100 would mean the recording would start 100 pixels toward the center).
Size is input as Width x Height (standard resolution notation), so you would input, say, 1920x1080 in there to record an area of 1920x1080. If you do it this way, you must manually hit the "Apply" button to set it.
While the default recording area is the Desktop (i.e. just the screen), you can also click and drag "Drag Here" to record various applications automatically (such as SC2).
You can see that "Drag Here" becomes red, denoting that you are currently dragging the application selector. Hitting "Desktop" will revert it to the default screen capture.
Step 4: Clicking the "Layout" button opens up the following window, which is used to control the screen regions being recorded.
What shows up is the layout of the regions that you are recording. Each region is, by default, surrounding in a green border with the output resolution in the top left corner. Those regions are resizeable and moveable, so you can move around the layout of the various regions you have (pretty much identical to how XSplit handles multiple camera sources).
You can easily record multiple screen regions for Picture-In-Picture by going to "Panel" and selecting "Add".
In the above, you can see that there are two regions being recorded. Each of the regions are separate, and can be manipulated in the same way a single region can be. "Panel -> Delete" deletes the currently selected region (which you can switch either by using the tabs at the top or by clicking on the regions themselves).
You can also edit each screen region individually in the main UI (i.e. you can select areas to record with each individual region, switching via the tabs at the top, etc.).
Note that having too many screen regions at the same time will significantly hurt your performance (it seems you can add as many regions as you want).
Step 5: Now that you finally have SCFH configured/learned, you can edit the values in FME 2.5. The only thing that differs between using VHScreenCap and SCFH is that you just need to select SCFH as the camera source. Also, there is no need to do any cropping at all. Simply select the output resolution under "Size", and SCFH will automatically handle all input resolutions. It will automatically change to reflect changes in FME while the two processes are linked.
Step 6: To stream, simply hit "Start". However, for a more economic method, close out of FME 2.5 (which will make SCFH freak out, but don't worry) and restart SCFH. Locate "FMEcmd.exe" in the installation folder of FME 2.5, and you will automatically start streaming. Hit Refresh in SCFH and select FMEcmd.exe, and you are now in control of the camera.
FMEcmd.exe takes fewer resources, but still maintains all functionality. Furthermore, you can change camera regions on-the-fly (that's the whole point!). Unfortunately, you cannot record via your webcam (or select audio options) with SCFH. To do that, you would need to use VHMultiCam, Virtual Audio Cable, etc.
In conclusion, I would like to say SCFH + VHMultiCam + VAC + FMLE is certainly no replacement for XSplit. It is a much more complicated process, and requires more work to set-up. However, it can serve as a nice substitute. Happy streaming!