Missing values cleanup in YouTubers landscape!¶

Creating the channel!¶

In [1]:

#load the dataset 
import pandas as pd
df = pd.read_csv('preprocessed_youtube_data.csv')
df.head()

Out[1]:

	Rank	Channel Name	Category	Subscribers	Country	Average Views	Average Likes	Average Comments	Content Type
0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	1.0	T-Series	Music & Dance	258400000.0	India	135200.0	5600.0	223.0	NaN
2	2.0	MrBeast	Video games	236100000.0	United States	104000000.0	4000000.0	74000.0	Humor
3	3.0	Cocomelon - Nursery Rhymes	Education	171400000.0	NaN	5100000.0	57100.0	0.0	NaN
4	4.0	SET India	NaN	167100000.0	India	27900.0	996.0	7.0	NaN

In [3]:

# Run the below cells to know more about the dataset structure!

In [2]:

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1046 entries, 0 to 1045
Data columns (total 9 columns):
 #   Column            Non-Null Count  Dtype  
---  ------            --------------  -----  
 0   Rank              1045 non-null   float64
 1   Channel Name      1045 non-null   object 
 2   Category          695 non-null    object 
 3   Subscribers       1045 non-null   float64
 4   Country           853 non-null    object 
 5   Average Views     1045 non-null   float64
 6   Average Likes     1045 non-null   float64
 7   Average Comments  1045 non-null   float64
 8   Content Type      335 non-null    object 
dtypes: float64(5), object(4)
memory usage: 73.7+ KB

In [3]:

df.describe()

Out[3]:

	Rank	Subscribers	Average Views	Average Likes	Average Comments
count	1045.000000	1.045000e+03	1.045000e+03	1.045000e+03	1045.000000
mean	521.055502	2.246191e+07	9.935658e+05	4.331511e+04	1281.757895
std	298.709262	1.744217e+07	3.871218e+06	1.628324e+05	9028.006321
min	1.000000	1.250000e+07	0.000000e+00	0.000000e+00	0.000000
25%	262.000000	1.420000e+07	3.630000e+04	5.430000e+02	3.000000
50%	523.000000	1.740000e+07	1.806000e+05	3.800000e+03	63.000000
75%	784.000000	2.410000e+07	8.374000e+05	2.760000e+04	369.000000
max	1000.000000	2.584000e+08	1.040000e+08	4.000000e+06	235000.000000

Warm-Up Activities¶

1. Identify Missing Data¶

In [11]:

df.isnull().sum().sum()

Out[11]:

2. Missing Data Detective¶

In [7]:

df.isnull().sum().idxmax()

Out[7]:

'Content Type'

Getting Started¶

Before starting with other activities , let's create a heatmap to visualize the distribution of missing data across the dataset

In [8]:

# Run the cell below

In [8]:

df.isnull()

Out[8]:

	Rank	Channel Name	Category	Subscribers	Country	Average Views	Average Likes	Average Comments	Content Type
0	True	True	True	True	True	True	True	True	True
1	False	False	False	False	False	False	False	False	True
2	False	False	False	False	False	False	False	False	False
3	False	False	False	False	True	False	False	False	True
4	False	False	True	False	False	False	False	False	True
...	...	...	...	...	...	...	...	...	...
1041	False	False	True	False	False	False	False	False	True
1042	False	False	False	False	False	False	False	False	False
1043	False	False	True	False	False	False	False	False	True
1044	False	False	False	False	False	False	False	False	True
1045	False	False	False	False	False	False	False	False	True

1046 rows × 9 columns

In [9]:

import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(12, 6))
sns.heatmap(df.isnull(), cbar=False, cmap='viridis')
plt.title('Heatmap of Missing Data')
plt.show()

3. Non-Missing Notifiers: Counting Valid Data¶

In [13]:

df['Subscribers'].notnull().sum()

Out[13]:

4. Remove Completely Empty Rows¶

In [17]:

df_cleaned = df.dropna(how='all')

In [18]:

df_cleaned.info()

<class 'pandas.core.frame.DataFrame'>
Index: 1045 entries, 1 to 1045
Data columns (total 9 columns):
 #   Column            Non-Null Count  Dtype  
---  ------            --------------  -----  
 0   Rank              1045 non-null   float64
 1   Channel Name      1045 non-null   object 
 2   Category          695 non-null    object 
 3   Subscribers       1045 non-null   float64
 4   Country           853 non-null    object 
 5   Average Views     1045 non-null   float64
 6   Average Likes     1045 non-null   float64
 7   Average Comments  1045 non-null   float64
 8   Content Type      335 non-null    object 
dtypes: float64(5), object(4)
memory usage: 81.6+ KB

5. Column-Specific Removal¶

In [21]:

0.5 * len(df)

Out[21]:

523.0

In [23]:

threshold = 0.5 * len(df)
df_cleaned_column = df.dropna(axis = 'columns', thresh = threshold)

6. The Rank Rescuer¶

In [26]:

df['Rank'] = df['Rank'].fillna(value = df['Rank'].mean())

In [25]:

Out[25]:

521.0555023923445

7. Use of Mean Imputation¶

In [27]:

# Run the below cell to understand the composition of Average Comments

In [28]:

import matplotlib.pyplot as plt
import seaborn as sns
plt.figure(figsize=(10, 6))
sns.boxplot(x=df['Average Comments'])
plt.title('Boxplot of Average Comments')
plt.xlabel('Average Comments')
plt.axvline(x=50000, color='r', linestyle='--', label='Outlier Threshold')
plt.show()

The boxplot shows a significant concentration of 'Average Comments' values within a narrow range, with a few videos having an abnormally high number of comments, up to 200,000.

8. Comment Caution: Imputing Average Comments¶

In [29]:

#Apply Median Imputation
df['Average Comments'] = df['Average Comments'].fillna(value=df['Average Comments'].median())

In [ ]:

9. Forward Fill for 'Country'¶

In [31]:

#Apply Forward Fill 
df['Country'] = df['Country'].ffill()

In [ ]:

In [37]:

# Run the below cell to check the distribution after forward fill

In [33]:

plt.figure(figsize=(10, 6))
sns.countplot(y=df['Country'], order=df['Country'].value_counts().index)
plt.title('Distribution of Country After Forward Fill')
plt.xlabel('Count')
plt.ylabel('Country')
plt.show()

10. Apply backward fill¶

In [34]:

#Apply Backward Fill
df['Country'] = df['Country'].bfill() 

In [ ]:

In [43]:

# Let's re run the cell again to see the distribution now

In [36]:

plt.figure(figsize=(10, 6))
sns.countplot(y=df['Country'], order=df['Country'].value_counts().index)
plt.title('Distribution of Country After Both Fill')
plt.xlabel('Count')
plt.ylabel('Country')
plt.show()

11. Combining Forward and Backward Fill¶

In [ ]:

12. View Boost: Interpolating Average Views¶

In [37]:

#Apply interpolation technique 
df['Average Views'] = df['Average Views'].interpolate()

In [ ]:

13. Impute Subscribers Using Forward Fill and Mode¶

In [ ]:

In [43]:

# Apply forward fill
df['Subscribers'] = df['Subscribers'].ffill()

In [44]:

# Now apply mode() and store the results in `mode_subscribers`
mode_subscribers = df['Subscribers'].mode()

In [45]:

# Now uncomment and run this cell
df['Subscribers'] = df['Subscribers'].fillna(mode_subscribers)

14. Impute Category Using Unknown¶

In [ ]:

# Run the cell below to understand which are the famous categories of channel watched on YouTube! 

In [47]:

plt.figure(figsize=(12, 6))
sns.countplot(x=df['Category'], order=df['Category'].value_counts().index, palette='viridis')
plt.title('Distribution of Categories', fontsize=16, fontweight='bold')
plt.xlabel('Category', fontsize=12)
plt.ylabel('Count', fontsize=12)
plt.xticks(rotation=45, ha='right')  
plt.tight_layout() 
plt.show()

The bar chart shows that Music & Dance, Movies, and Animation have the highest counts, making them the most common categories in the dataset.

In contrast, categories like Health & Self Help and Fashion have much lower counts, indicating they are less common.

In [49]:

df['Category'].isnull().sum()

Out[49]:

In [50]:

# Run the cell below to understand the missing value percentage among all categories 
category_counts = df['Category'].value_counts()
category_percentage = category_counts / len(df) * 100

# Check the number of missing values
missing_values_count = df['Category'].isnull().sum()
missing_values_percentage = missing_values_count / len(df) * 100

# Print results
print("Category Distribution:\n", category_counts)
print("\nCategory Percentage:\n", category_percentage)
print("\nMissing Values in 'Category':", missing_values_count, "(", missing_values_percentage, "%)")

Category Distribution:
 Category
Music & Dance           210
Movies                  157
Animation               133
Daily vlogs              37
News & Politics          31
Video games              29
Education                21
Toys                     14
Humor                    13
Food & Drinks            12
Sports                    8
Science & Technology      8
Fitness                   7
DIY & Life Hacks          3
Design/art                2
ASMR                      2
Autos & Vehicles          2
Beauty                    1
Travel                    1
Health & Self Help        1
Animals & Pets            1
Fashion                   1
Mystery                   1
Name: count, dtype: int64

Category Percentage:
 Category
Music & Dance           20.076482
Movies                  15.009560
Animation               12.715105
Daily vlogs              3.537285
News & Politics          2.963671
Video games              2.772467
Education                2.007648
Toys                     1.338432
Humor                    1.242830
Food & Drinks            1.147228
Sports                   0.764818
Science & Technology     0.764818
Fitness                  0.669216
DIY & Life Hacks         0.286807
Design/art               0.191205
ASMR                     0.191205
Autos & Vehicles         0.191205
Beauty                   0.095602
Travel                   0.095602
Health & Self Help       0.095602
Animals & Pets           0.095602
Fashion                  0.095602
Mystery                  0.095602
Name: count, dtype: float64

Missing Values in 'Category': 351 ( 33.55640535372849 %)

In [ ]:

In [51]:

df['Category'] = df['Category'].fillna(value='Unknown')

In [52]:

df['Category']

Out[52]:

0             Unknown
1       Music & Dance
2         Video games
3           Education
4             Unknown
            ...      
1041          Unknown
1042        Animation
1043          Unknown
1044    Music & Dance
1045            Humor
Name: Category, Length: 1046, dtype: object

15. Fill Missing Content Type Values with Unknown¶

In [54]:

df['Content Type'] = df['Content Type'].fillna(value='Unknown') 

In [ ]:

Final Review of Missing Values¶

The result should match the following output:

Final_Review

In [68]:

# Run the below cell to check for final missing values

In [ ]:

missing_data_final = df.isnull().sum()
print(missing_data_final)

Moving Forward!¶

In [ ]:

Statement of Completion#c2fc7008

Data Cleaning with Pandas

Handling Missing and Null Values in YouTube Channel Data

Missing values cleanup in YouTubers landscape!¶

Creating the channel!¶

Warm-Up Activities¶

1. Identify Missing Data¶

2. Missing Data Detective¶

Getting Started¶

3. Non-Missing Notifiers: Counting Valid Data¶

4. Remove Completely Empty Rows¶

5. Column-Specific Removal¶

6. The Rank Rescuer¶

7. Use of Mean Imputation¶

8. Comment Caution: Imputing Average Comments¶

9. Forward Fill for 'Country'¶

10. Apply backward fill¶

11. Combining Forward and Backward Fill¶

12. View Boost: Interpolating Average Views¶

13. Impute Subscribers Using Forward Fill and Mode¶

14. Impute Category Using Unknown¶

15. Fill Missing Content Type Values with Unknown¶

Final Review of Missing Values¶

Moving Forward!¶