Association and Demographics Based Recommended System

Summary

The lecture explains three connected topics: association-based recommendation, demographic-based recommendation, and website personalization.

1. Association-Based Recommender Systems

Association-based recommendation applies association rule mining to find items that frequently occur together. The approach is similar to market-basket analysis, but a “transaction” can represent items users viewed, searched for, or purchased together. Co-viewing can be useful even when users do not buy online—for example, they may purchase an item later in a physical store.

A rule has the form X → Y: when a transaction contains item set X, it may also contain item set Y.

Two common measures are used to assess rules:

• Support: How often X and Y occur together across all transactions. In probability terms, this is the probability that a transaction contains both.
• Confidence: How often Y occurs in transactions that contain X. It is calculated as the number of transactions containing both X and Y divided by the number containing X.

Minimum support and confidence thresholds help filter out rules that are too rare or unreliable. For example, the lecture describes a rule where A and D occur together in 60% of transactions; A → D has 100% confidence, while D → A has 75% confidence.

Apriori Method

The lecture demonstrates the Apriori algorithm as one way to find frequent itemsets and derive rules. Its key principle is that if an itemset is not frequent, then any larger itemset containing it cannot be frequent.

The process is:

1. Scan transactions for individual items and count how often each occurs.
2. Prune infrequent items using the minimum support threshold.
3. Join the remaining frequent itemsets to form candidate itemsets of the next size.
4. Scan the transaction data again to count each candidate’s occurrences.
5. Prune candidates that do not meet minimum support.
6. Repeat joining and pruning until no larger frequent itemsets remain.
7. Generate association rules from the frequent itemsets.
8. Calculate rule confidence and retain rules that meet the minimum confidence threshold.

To recommend items, find rules whose left-hand side matches items in the active user’s current basket or session. Suggest the unique items on the right-hand sides, typically prioritizing rules with higher confidence. This method is intended for item recommendations, not for ranking or predicting users.

Apriori can be inefficient at large scale because it repeatedly scans the transaction database. That cost can become substantial when there are millions of transactions or many co-viewed items.

2. Demographic-Based Recommender Systems

A demographic-based system recommends items or categories based on the preferences of users with similar demographic characteristics. Unlike association rules, which recommend at the item level, this approach often predicts preferences at the category level.

Grouping related items into categories—for example, pens and pencils under office stationery—can reduce sparsity. Two demographically similar users may not have interacted with the same specific products, but they may share preferences for broader categories.

A typical process is:

1. Prepare training examples using user demographics as input attributes and category preferences as the target.
2. Convert item-level behavior into category-level preferences. Preferences may be represented as binary values, such as whether the user has shown enough interest in a category to pass a chosen threshold.
3. Train a predictive model to estimate category preferences from demographics. The lecture mentions general-purpose methods such as decision trees and artificial neural networks.
4. Apply the trained model to a new user’s demographics to predict preferred categories.
5. Select the strongest predictions to produce recommendations. The system can choose top categories or items for one user, or identify users most likely to prefer a particular category.

3. Website Personalization

Recommender-system methods can also personalize websites by reorganizing pages or adapting their content, structure, layout, or media format.

Personalization may be:

• User-directed: The user chooses what appears on the page.
• Automated: The system adapts the site based on the user’s observed behavior and patterns among other users.

Potential benefits include improving site usability, converting visitors into buyers, retaining existing customers, and supporting cross-selling, upselling, and targeted advertising.

A general personalization workflow is:

1. Collect relevant data, which may include user demographics, user behavior, and environmental information.
2. Gather data either explicitly (a reactive approach) or implicitly from observed behavior (a non-reactive approach).
3. Preprocess the data and build user profiles using data-mining methods such as classification.
4. Use the resulting models to produce a personalized website experience.

Speakers and Sources Featured

• Speaker: One instructor or narrator; no other speakers are identifiable in the subtitles.
• Examples and data contexts referenced: Market-basket transactions, access-log and browsing-session data, e-commerce views and purchases, user-item rating or behavior data, demographic data, and item/category information.