Humm... you are looking for a product recommendation engine then... Well, they come, basically, in three flavours:
The first one gathers and stores data on your users' activities, preferences, behavior, etc... This data is then sent into an engine that separates it into user channels. Each channel has certain characteristic likes and dislikes. So, when you have a new visitor he or she will be classified and be assiged an specific user profile. Then items will be displayed based on this profile's likes/dislikes.
Now, content-based filtering uses a different approach - a less social one - by taking into account ONLY your user's previous browsing history, his preferences and activities. Essentially, this will create recommendations based on what this user has previously liked/purchased.
But why choose just one of them, right? Hybrid recommender systems uses a bit of both to provide a personalized yet social recommendation. These are usually more accurate when it comes to providing recommendations.
I think that the collaborative filtering is a great option when you have a big influx of users - it's kinda hard to build good channels with only 42 users/month accessing your website.
The second option, based on content, is better for a small site with plenty of products - however, IMHO, the third one is the one for you - build something that will get users going from the start and gather all that data they generate to, in the future, be able to offer a amazon-like recommendation experience!
Building one of these is no easy task as I'm sure you already know... but I strongly recommend this book (using a personal-history filtering!) which has really came through for me in the past: http://www.amazon.com/Algorithms-Intelligent-Web-Haralambos-Marmanis/dp/1933988665
Good luck and good learning!
Two books that cover recommender systems:
Basically two ways of approaching the problem, user or item based. Netflix appears to use the former, while Amazon the latter. Typically user based requires more time and/or processing power to generate recommendations because you tend to have more users than items to consider.
While it doesn't talk about advertising directly, the book Algorithms of the Intelligent Web discusses several topics that I would imagine would be very valuable in setting up an advertising system - especially concerning how/where to place ads and how to target viewers (provide ad content tailored to a viewer or to a specific webpage with a certain class of viewers in mind). These include:
You may want to also check out Collective Intelligence in Action. It discusses things like:
The answers you are looking for really can not be found using lucene alone. You need ranking and grouping algorithms to filter and understand the data and how it relates. Lucene can help you get normalized data, but you need the right algorithm after that.
I would recommend you check out one or all of the following books, they will help you with the math and get you pointed in the right direction:
Algorithms of the Intelligent Web
Collective Intelligence in Action
Programming Collective Intelligence