No doubt in that, use OpenCV.
But remember, you have a long way to go.
1. First of all, you should be good in C++ and object oriented programming.
Well, if you are not good, try to learn it first. Check out following link for some best resources : https://stackoverflow.com/questions/909323/what-are-good-online-resources-or-tutorials-to-learn-c
2. Then get OpenCV and install
Check out OpenCV homepage to get info about downloading and installing OpenCV.
3. Now get and read some good books on OpenCV
The best book on OpenCV is "Learning OpenCV" written by Gary Bradsky, main founder of OpenCV.
Second one is "OpenCV cookbook".
These books contains lots of examples on OpenCV along with description
4. Check out OpenCV documentation.
OpenCV documentation contains details of complete functions. It also includes a lot of tutorials, which are really good for all.
5. Also try running OpenCV samples. It contains a lot of good programs
And always, Google is your best friend. Ask everything there first. Come here only when you are lost in your path.
Acquire all the above things. Then you will be really good in OpenCV and i am sure you will enjoy its power. Once you are done with these, you will get enough idea on realizing your project.( Otherwise, you will post new questions every day asking codes to realize your project, which will be useless for you. )
For your understanding, your project include advanced things like Optical Character Recognition. It is a big topic. So build yourself from basics. And it will take time.
All the best.
You can use image subtraction. Cross-correlation is also acceptable here.
Google search phrase: background subtraction algorithm.
background subtraction algorithm
Also this book contains needed info for you.
openCV is probably the most complete free image processing library.
There is also a book which describes both the library and some image processing techniques.
This is a reasonably complex problem, not exactly graduate research but challenging!
See this question for a list of other books.
1. Get OpenCV
Check out OpenCV homepage to download OpenCV source.
2. Check out this SOF for more details on OpenCV on iOS
iPhone and OpenCV
3. Get and read some good books on OpenCV
Have you taken a look at the camshift paper by Gary Bradski? You can download it from here
I used the the skin detection algorithm a year ago for detecting skin regions for hand tracking and it is robust. It depends on how you use it.
The first problem with using color for tracking is that it is not robust to lighting variations or like you mentioned, when people have different skin tones. However this can be solved easily as mentioned in the paper by:
Throwing away the V channel in HSV and only considering H and S channels is really enough (surprisingly) to detect different skin tones and under different lighting variations. A plus side is that its computation is fast.
These steps and the corresponding code can be found in the original OpenCV book.
As a side note, I've also used Gaussian Mixture Models (GMM) before. If you are only considering color then I would say using histograms or GMM makes not much difference. In fact the histogram would perform better (if your GMM is not constructed to account for lighting variations etc.). GMM is good if your sample vectors are more sophisticated (i.e. you consider other features) but speed-wise histogram is much faster because computing the probability map using histogram is essentially a table lookup whereas GMM requires performing a matrix computation (for vector with dimension > 1 in the formula for multi-dimension gaussian distribution) which can be time consuming for real time applications.
So in conclusion, if you are only trying to detect skin regions using color, then go with the histogram method. You can adapt it to consider local gradient as well (i.e. histogram of gradients but possibly not going to the full extent of Dalal and Trigg's human detection algo.) so that it can differentiate between skin and regions with similar color (e.g. cardboard or wooden furniture) using the local texture information. But that would require more effort.
For sample source code on how to use histogram for skin detection, you can take a look at OpenCV"s page here. But do note that it is mentioned on that webpage that they only use the hue channel and that using both hue and saturation would give better result.
For a more sophisticated approach, you can take a look at the work on "Detecting naked people" by Margaret Fleck and David Forsyth. This was one of the earlier work on detecting skin regions that considers both color and texture. The details can be found here.
A great resource for source code related to computer vision and image processing, which happens to include code for visual tracking can be found here. And not, its not OpenCV.
Hope this helps.
I would recommend a book by OpenCV author Gary Bradski - Learning OpenCV: Computer Vision with the OpenCV Library.
It is not only a refence how to use OpenCV, but also a comprehensive book on many computer vision topics, with many images that illustrate the concepts. It guides you through OpenCV basics, then trough image processing using filters, convolutions, histograms, contours, segmentation, tracking, camera calibration and 3D vision.
It is relatively easy to read (not too much math, just enough), I liked it really much.
There is an O'Reilly book by two of the major authors of OpenCV: Learning OpenCV: Computer Vision with the OpenCV Library by Bradski and Kaehler.
You should note that it is based on OpenCV 1.0, not the more recent 2.2.
Nonetheless, to understand algorithms it will likely be useful. For most of the higher-level algorithms like corner detection, for example, the book contains a mathematical descriptions of the implementation. Also, the authors do a decent job of providing references to academic journal articles, so even if their own description of the implementation is lacking, you will be able to use the references as a starting point.