Video summary
Using the Web log : Web usage mining
Main summary
Key takeaways
Summary
The lecture explains how cleaned web-server logs can be analyzed to understand visitor behavior, improve websites, and support business decisions. Access logs record navigation requests, but they must first be cleaned and organized into sessions before meaningful analysis can be performed.
Web-log analysis workflow
- Gather data: Use access logs together with information about site content and, where available, the sitemap or site dictionary.
- Clean and prepare the logs: Use site information to help identify and address problems in the raw data.
- Form sessions: Apply heuristics—such as time-based rules and relationships between pages—to group requests into visits. Identifying a visitor consistently is difficult; a timeout of around 30 minutes is commonly used as one session rule.
- Create structured data: Produce flat files with fields suitable for analysis.
- Analyze behavior: Use the prepared data to study page access, customer behavior, product browsing, and ordering.
- Store and explore combined data: Session and content data can be held in a data warehouse or data mart for multidimensional analysis and data mining.
Main analysis approaches
-
Static aggregation: Summarizes log data by predefined units, such as day or session, to produce reports and general statistics.
- It is relatively fast, requires little processing, and needs minimal disk space.
- It provides a quick overview—for example, page views, sessions that included a particular page, or shopping-cart use.
- It offers limited insight into deeper patterns or the reasons behind user behavior.
-
Session analysis: Examines requests grouped into individual sessions or groups of sessions.
- It can reveal how visitors move through the site and help identify navigation problems, such as pages visitors commonly try to use together that are not properly linked.
- Its usefulness depends on sound session-building heuristics and appropriate behavior models. Crude grouping makes it difficult to generalize findings.
-
OLAP (multidimensional analysis): Lets analysts explore data at progressively more detailed levels—for example, narrowing from product browsing to a category and then to specific products.
-
Data mining: Applies algorithms to discover patterns or make predictions from the prepared data.
Common measures and terms
- Hit: A successful server request for a resource. A hit does not necessarily mean a page view, since a page can also load embedded resources.
- Page view: A request for a resource defined as a page, excluding embedded resources.
- Visit/session: A series of requests attributed to the same identified client within session rules, often using a timeout such as 30 minutes. A visit can contain one or more page views.
- Click path: The ordered sequence of hyperlinks or pages a visitor follows. Log data is also commonly called clickstream data.
- Trail: The time-ordered sequence of web pages visited during a session.
Mining methods and uses
Different tasks call for different analytical methods:
- Sequence mining or Markov-chain analysis: Predict likely next actions or pages and analyze navigation sequences.
- Association-rule mining: Find pages or products that tend to occur together.
- Clustering: Group visitors or sessions by shared characteristics or behavior.
- Classification: Assign visitors or events to predefined categories, such as identifying potentially fraudulent transactions.
The lecture uses association-rule mining to illustrate how frequently co-visited pages can be found. It notes that this method also has broader applications, including market-basket analysis.
Practical applications
Web-usage mining can support:
- Page prefetching and caching.
- Website reorganization and personalization.
- Recommendations for links and products.
- Predictions about the next page a visitor may request.
- Cross-selling and upselling by connecting pages or site objects to products and identifying related items.
- Improved search quality and evaluation of search results.
- Increased site usability and business performance.
The central idea is to turn clickstream data into useful patterns and decisions. The lecture describes web-usage mining as a way to build more personalized, user-friendly, and commercially effective websites.
Speakers and sources featured
- Speaker: One lecturer/narrator; no other speakers are identifiable in the subtitles.
- Data sources and tools mentioned: Web-server access logs, sitemaps, site content or site dictionaries, data warehouses and data marts, Google Analytics, Webalizer, Sawmill, WebTrends, AWStats, WWW Stats, and Apache Log Viewer.
Rate this summary
Your feedback will help improve summaries.
Improve this summary
Reprocess with a stronger model when the summary feels incomplete or inaccurate.
Translate summary in another language
Ask questions to this video
Chat for follow-up questions, clarifications, and source-backed answers.