I just finished a book titled Super Crunchers, written by Ian Ayres. The theory of the book is that large-dataset analysis is fast becoming the way to make decisions in a variety of fields, from Web analysis to purchasing to criminal justice. Call it the end of human intuition, but the notion is not as overwheming as you might think. The concept of data mining is not new and over the past ten years several major breakthroughs make it possible for real people (like me) to engage in number crunching on the piles of data we keep. All done through smaller, faster computers, more access to data than ever before and easy-to-use analytical software that doesn’t compromise rigor for speed.

An everyday example of this type of data analysis is the kind of recommendations each of us is likely to see each day. Consider some of the following:

• If you are a Netflix user you are given recommendations every time you add a movie to your queue. And, you help their recommendation engine each time you rate your recent selections.
• If you are an Amazon user, you receive recommendations based on your past purchases. And, while this started with books and was relatively predictable, Amazon continued to perfect their recommendations. Simply because I bought The Elements of Style ten years ago, I today get more than just suggestions for that latest text on English grammar and usage.
• If you are a ebags.com user you are asked to rate your purchases. These ratings are fed back to the buyers and product developers at bags.com so they can continue to buy and deliver the high-quality products their customers want.

Everywhere you look recommendations are at work, and job searching is no exception. The CareerBuilder.com recommendation engine is at work in two very specific and powerful ways:

• The first is when a person looks for a job on CareerBuilder.com. Using our exclusive SmartMatch technology, we define a user profile from the activity and serve recommendations of alternate jobs to that job seeker, aiding and expanding his or her job search.
• Secondly, the same engine, when paired with the Resume Database, is used to assist employers identify like candidates. R2 or Recommended Resume is a patent-pending matching technology that enables employers to save time finding the right candidate for any given job. Once a job is given certain criteria, the context-sensitive engine finds candidates and sorts applicants based on the desired requirements to find the 50 closest-matching resumes that match a Job Posting or match another resume.

Both techniques streamline the job search or hiring process – and, unlike our competition, we’ve been recommending jobs and potential employees for the past five years. This is exactly what we are supposed to use all this technology for, isn’t it? To make our lives a bit easier and more streamlined so that we can focus on other stuff – like filling those open positions.

If you haven’t spent much time thinking about the application of data-mining at your company or industry, you might want to pick up Super Crunchers or watch the video below. I guarantee there are loads of smart folks waiting to help you unlock the potential in your data.