A reliable and trusted source for learning about the latest database marketing and analytical tools, Joe Weissmann is often among the first to see the link between direct marketing and what is happening in other disciplines. Now president of Culver City, CA-based JWDirect, a Web services supplier he formed in 1992, Weissmann founded Demographic Research in 1975, after six years with ADVO and Metromail (now Experian). Weissmann has been at the forefront of database marketing and statistical analysis for nearly 30 years; his latest focus is on support vector machine, or SVM, analysis.
While far from a household word in direct marketing circles, SVM is not a brand-spanking new concept. It is a statistical modeling methodology that has proved effective in a variety of fields, primarily those encompassing a large number of variables, such as face-recognition, bioinformatics (gene classifications), and the classification of Web pages for search engine companies such as Google.
Losing — and finding — your marbles
To explain how SVM works, think of the following analogy. Suppose you were told that someone had taken a bag of children’s marbles and scattered them along the boundary between two properties in a square dirt field, 100 yards on a side. Your task was to take a map of the field and draw a reasonably accurate boundary line, based on the trail of marbles.
Now, you don’t know exactly how many marbles there are, but you can guesstimate by assuming that the bag is neither so small as to contain fewer than a handful nor so large as to contain more than a few handfuls (“few” being relative, of course). You assume that the marbles are all perfectly round and contain the typical combination of colors and patterns you would expect to find in a bag of children’s marbles.
How would you go about accomplishing the task? I’ll bet you would walk around the edge of the field, hoping to sight the trail of marbles there somewhere, and then follow that trail, making marks on the map where you find each marble. If you reach the other side of the field, you would have nearly completed the task; you’d need only to ascertain and draw the most reasonable line that fit the pattern of marks on your map.
You would probably look for the trend in the line as well, creating as straight a boundary as you could (not zigging or zagging in connect-the-dots fashion as a literal track-the-marbles line would do). If you found a few stray marbles outside of your trend line, you would probably just ignore them.
But what if the trail — which had marbles every two or three feet — sort of curved off 20 yards from the far edge of the field and then you couldn’t find any more marbles nearby, but you found one or two at the edge of the field? If you had collected about a “bagful” of marbles, you might assume that the gap was there because there weren’t enough marbles to complete the line. If you didn’t have a bagful, you might go back to see if there were a major deviation in the straight line at the point where the trail stopped.
The point is, because you know from experience what a marble looks like, you don’t have to explain to someone how to differentiate a marble from stones of all shapes, colors, and sizes; coins; bottlecaps; glass fragments, gumballs, cigarette wrappers; or any other debris that might be in the field. You also have a sense that the boundary line is probably relatively straight, or that at least it is not randomly meandering, and it doesn’t go in a loop-de-loop pattern.
In other words, using common sense and intuition, you go out and pick up the marbles, make your marks, and finish the job in good order.
Focusing on the margins
Another reason you’d get the job done effectively is that you focused your attention on where the marbles were, and once you found the beginning of the trail, you went forward where you expected them to be. Going marble by marble, you might have discovered that the trail actually curved and nearly looped back around to its beginning, rather than cutting straight across the field, but you’d have had little trouble following the marble markers, so long as they were distributed more or less evenly, without major gaps. If you saw several bunched together, you’d probably have made a single mark, picked them up, and moved on.
As noted, this is all common sense. And it happens to be how support vector machines work. But it is not at all how your typical statistical analysis works. To complete the analogy, with most other statistical methods you would have to perch a robot high above the field on a platform, teach the robot to distinguish between marbles and other objects such as stones, then have the robot look through binoculars to find each of the marbles. If there were a gravel patch in the field, you’d have no trouble finding the marbles, but your robot probably would!
This analogy illustrates how SVM focuses on “the margin” — the relationship that each point (or marble) has to the boundary line between two spaces — and how it focuses on the points closest to the boundary. These closest points are called support vectors, hence the name support vector machine analysis.
Standard statistical methods, whether regression, decision trees, or neural network analysis, are not designed to focus in the way that SVM methods are. The more the marbles in the analogy zig and zag, the more difficult it is to find a trend line. By trying to find as many support vectors as possible, SVM is based on a much more practical relationship between the vectors and the trend line than are methods that examine the relationships among the points before attempting to establish a trend line. SVM behaves as you would — by focusing on what you intuitively expect a boundary line to be.
To use a different analogy entirely, while standard statistical methods have trouble seeing the forest for the trees, SVM sees both quite clearly.
SVM is a form of data mining; it’s also a form of artificial intelligence applied to data mining. By combining the two, bringing “pattern recognition” and “statistical probability” into the picture. SVM can handle a much larger number of inputs than traditional mining as well. And it knows when “close enough” is enough.
Direct marketing applications
In direct marketing, you are trying to draw a line (or make a prediction to distinguish) between those who will respond to an offer and those who won’t, based on the characteristics of the offer and the characteristics of the target universe. If you want to give a weight to the prediction, you can determine how far from the boundary line a specific point, vector, or prospect is.
Modeling is the process of using historical or “training” data — in which the response results are known — to determine where the points are that best fit a boundary line. With methods other than SVM, there is a tendency to “overfit” the training data, so that while the model is extremely accurate in explaining what happened before, it is misleading in predicting what will happen in the future under similar but different circumstances.
SVM allows you to make accurate predictions of future behavior based on a practical analysis of past behavior. To use still another analogy, instead of categorizing all the brush strokes in a fine-art portrait, it gives you a paint-by-numbers template to reproduce it. The result will be recognizable and serve a useful purpose.
SVM happens to be based on logic that follows the precepts of Ockham’s Razor, or the so-called Law of Economy, stated by William of Ockham in the 14th century, which says that a simpler solution is preferable to a more complex one if the results are the same.
So far, JWDirect is the only organization to apply SVM to direct marketing. Joe Weissmann is offering a shared-risk guarantee: If SVM modeling fails to work as promised, JWDirect will refund two-thirds of its $15,000 modeling fee. After the first half-dozen successful clients use SVM, the modeling fee will be increased to $25,000 (without the refund option). Clients pleased with their SVM model can have a customized system installed inhouse for $50,000, which includes training on using it.
Ernie Schell is president of Marketing Systems Analysis, a Southampton, PA-based consultancy.