GCI World September 2026 Session 3

Main ideas, concepts, and lessons

Course logistics & how to participate

This session is Lesson 3, focused on data cleaning using pandas (Python).

Participants are reminded to:

• Complete attendance surveys and homework by the deadlines.
• Use official resources for “what to do next,” primarily the student handbook.
• Seek help from GCA World staff, colleagues, and office hours (recommended—many assistants are GCI graduates).

Tools used throughout the course (standard workflow)

The standard workflow uses:

• Omni Campus: course registration and assignment submission
• Slack: announcements and peer/assistant Q&A
• Student guide: lecture info and access points
• Google Drive: lecture materials (slides, notebooks, assignments)

Participants are asked to bookmark these and can access via QR code / Slack link.

Community guidance / rules

• No official WhatsApp groups are run by organizers.
• Any WhatsApp groups are member-created, and organizers are not responsible.

Participant setup requirement

• Ensure your Slack display name matches your Omni Campus account name so your results are tracked correctly.

Office hours

• Multiple regular office hours across different days/time zones.
• Special consulting hours are hosted by outstanding past GCI World students, potentially in their native languages.

Deadline change for attendance & homework

Original due time:

• 2 weeks after lecture at 11:00 AM UTC

New due time:

• 2 weeks + 1 day after lecture at 1:00 AM UTC

Examples:

• Attendance for first session: Oct 8 (11:00 AM) → Oct 9 (1:00 AM UTC)
• Homework for second session: Oct 8 (11:00 AM) → Oct 9 (1:00 AM UTC)

The course emphasizes extended work time.

---

Lecture content: Pandas for data cleaning (pandas fundamentals + practical tasks)

1) Where pandas fits in a data project workflow

A typical data processing project has 5 stages:

1. Collection
2. Cleaning
3. Exploration
4. Modeling
5. Production/results

This session emphasizes pandas for collection/cleaning/exploration, especially:

• Handling inconsistent formats
• Fixing anomalies
• Managing missing values

Key claim:

• ~80% of analysis effort may happen before modeling.

2) Structured vs unstructured data

• Structured/tabular data: 2D tables (Excel/CSV) → pandas focus
• Unstructured data: text/images/videos → not the focus here

3) What pandas is

Pandas is a Python library for working with tabular numeric data and more, supporting:

• Series & DataFrame structures
• Numeric operations (including aggregation)
• Reading/writing common formats (CSV, Excel, JSON)

Limitation mentioned:

• It can be slow/memory-heavy on very large datasets, though many cleaning tasks are still manageable.

4) Core pandas data structures

• Series (1D)
  • Like a labeled 1D array
  • Has values and an index (index labels can be customized)
• DataFrame (2D)
  • Table of rows and columns
  • Organized with:
    • row index
    • column labels
    • values

5) Using Google Colab + notebook execution

To open the course notebook in Colab:

• If it doesn’t open, try “Open with” → “Connect more apps” → add/select Google Colaboratory (Colab), then install/open.

Exercises run by executing notebook “cells” (e.g., Shift+Enter or the execute button).

---

Hands-on methodology & instruction-style workflow (pandas operations)

A) Importing pandas

Typical import:

import pandas as as pd

Using alias pd:

• e.g., pd.read_csv(...)

B) Creating and inspecting Series/DataFrames

Create a Series:

• pd.Series([...])

Access a Series:

• Values: series.values
• Index labels: series.index

Create a DataFrame:

• Build a dictionary where keys are column names and values are column data lists:
  • pd.DataFrame(dict_of_columns)

Inspect a DataFrame:

• df.head() (first 5 rows by default)
• df.info() (types and missingness)
• df.describe() (summary statistics)

C) Loading and using a real dataset (anime data)

Steps include:

• Download a ZIP dataset from a URL using:
  • requests to fetch
  • zipfile + I/O utilities to unzip
• Use the tree command to view extracted files.
• Load selected CSVs with pd.read_csv(...), then inspect using:
  • head, info, describe

D) Data extraction / selection (columns, rows, elements)

Column selection

• Single column: df['colname']
• Multiple columns: df[['col1', 'col2']]
• Dot notation may fail if column names contain spaces—use bracket notation.

Row selection by slicing

• NumPy-like end-exclusive slicing:
  • e.g., df[0:3]
• Note: label slicing behaves differently (end-inclusive for label-based slicing).

Label-based indexing

• Use df.loc[...] for:
  • selecting by row/column labels
  • label-based slicing (inclusive end)
• Can select:
  • rows only
  • columns only
  • row+column combinations in one call

Index/position-based indexing

• Use df.iloc[...] for selecting by integer positions

Single element access

• df.at[row_label, col_label]
• df.iat[row_index, col_index]

Updating values

• Assign replacement values using loc/at (or similar) for existing cells.

E) Organizing the mental model for selection methods

Selection can be understood using two dimensions:

1. Granularity: whole DataFrame vs single element
2. Reference type: labels vs integer indices

Mapping:

• loc = label-based
• iloc = position-based
• at/iat = single element (faster for scalars)

F) Creating/replacing columns and values

• Create a new column by assigning to a new column name (assignment creates it).
• Replace a specific cell using scalar selection methods such as df.at[...].
• A replace method exists (see docs for details).

G) Conditional filtering (subset rows)

General pattern:

• Put a boolean condition inside square brackets after the DataFrame.

Multiple conditions:

• Combine with logical operators (typically using parentheses):
  • (cond1) & (cond2)

Membership filtering:

• Use isin to keep rows where a value is in a set (e.g., studios in {Sunrise, Nomad}).

H) Practice filtering example (Lesson exercise)

Filter for:

• source == manga
• and members > 100000

Combine conditions using logical conjunction inside the filtering brackets.

I) Handling missing values (NaN)

• Missing values are represented as NaN.
• Check missingness:
  • df.isna()
• Count missing values per column:
  • use isna() + sum() over truth values

Dataset outcomes mentioned:

• anime.csv: no missing values
• anime_list.csv: no missing values
• anime_with_synopses.csv: 8 missing values in the synopsis column

J) Sorting

• Sorting by index (“index sort” / reorder by index)
• Sorting by column values:
  • sort_values(by='col', ascending=True/False)

K) Merging/joining multiple DataFrames

Merge using:

• pd.merge(...) (key-based join)

Key concept:

• A key is a column (or columns) that identifies rows.

pd.merge defaults:

• Inner join (only rows with matching keys in both tables are kept)

If key column names differ:

• Use left_on and right_on

Join types:

• Inner join: keep matching keys only
• Left outer join: keep all rows from left table; unmatched right rows become NaN
• Full outer join: keep all rows from both; unmatched entries become NaN

Emphasis:

• Correct key specification matters.
• Outer joins keep more data but introduce NaNs that must be handled.

L) Concatenation (stacking or side-by-side)

• pd.concat(...) combines DataFrames without key matching.

Default behavior:

• Vertical concatenation (stack rows)
  • Keeps original indexes (may cause duplicates)
  • Can reset via ignore_index=True
  • If columns don’t match, missing cells become NaN

Horizontal concatenation:

• Use axis=1 to join side-by-side based on index
• If indexes don’t align, NaNs can appear

M) Dropping data and duplicates

Remove using drop(...):

• Rows: provide index=...
• Columns: provide columns=... or axis=1

Reset index if needed:

• reset_index(...)

Duplicates:

• Identify with duplicated(...)
• Remove with drop_duplicates(...)
• May require resetting index afterward.

N) Mapping/transformation operations

• Map codes to labels:
  • create a lookup dictionary
  • use map() to convert numeric/state codes to readable strings
• More complex transformations:
  • use lambda with if logic (may include preprocessing such as type conversion)

O) Binning (discretizing continuous values)

• cut
  • split into equal-width intervals based on breakpoints
  • endpoint inclusion controlled by bracket/parenthesis semantics
• value_counts
  • count how many fall in each bin
• qcut
  • split into bins with equal counts (quantiles)

Decision principle:

• Choose based on goals (equal width vs equal frequency).

P) Aggregation and group operations

Use groupby:

• split DataFrame into groups based on one or more columns

Common operations:

• size() (count groups)
• mean() (average by group)
• agg([...]) (multiple aggregations: count/mean/max/min/etc.)

Note:

• groupby objects are iterable; looping yields each group.

Q) Missing data and outlier handling (basic overview)

Missing data basics for anime:

• Convert “unknown” values to a non-NaN representation (mentioned as a preprocessing step).

Strategies described:

• Listwise deletion: drop rows with any missing values
  • Example impact: ~17k rows → ~3k rows
• Pairwise deletion: ignore missing values per column and use available data (conceptually described)
• Imputation:
  • fillna(value) with a constant (example: -1)
  • Mean imputation possible but warned about bias and leakage risks
  • For time series: fill using previous time steps to avoid using future data (data leakage)

Outliers (conceptual overview):

• Visualize with box plots
• Clip at percentiles
• Or use normal-distribution assumptions and thresholds (e.g., outside 3 standard deviations)

Reminder:

• No single method fits all—choose based on data and goals.

---

Homework & attendance task structure (final instructions)

Task types

Two types of tasks:

• Attendance surveys
• Homework (Python code submitted to Omni Campus)

Attendance surveys

• Deadline for “today’s” survey: Oct 16, 1:00 AM UTC
• To complete the course:
  • submit at least 7 out of 14 surveys
• Surveys for first and second classes due: Oct 9, 1:00 AM UTC
• Late participants:
  • use lecture recordings via the student record
  • then submit attendance survey on Omni Campus

Homework

• Write Python code and submit to Omni Campus
• Deadline: Oct 16, 1:00 AM UTC
• Scoring:
  • On-time: 3 points
  • Late/missed: maximum 2 points
  • 8 homework assignments total → maximum 24 points
• Completion requirement:
  • need at least 14 points

---

Upcoming next lecture

Next week: Matplotlib (data visualization) Materials scheduled to be uploaded tomorrow.

---

Speakers / sources featured

Speakers

• Erin (member of Matsuo lab; welcomes participants, shares implementation tips, introduces the instructor)
• Akihiro “Aki” / Akihiro (postdoctoral fellow in Matsuo’s lab; main lecturer/instructor)

Assistant/consulting sources mentioned

• GCA World staff
• GCI graduates serving as office hour assistants
• Special consulting assistants: outstanding past students from multiple countries (Thailand, Vietnam, Indonesia, Malaysia, Colombia, Zimbabwe)

External data source referenced

• MyAnimeList database (anime dataset used in the Colab exercise)