1. Introduction¶
I'll work with TMDb_Movies Dataset, and try to answer the following questions;¶
- Data Cleaning - Filling zero Values
- what is the average budget
- Creating new features
- What are the variables that affect a film's revenue and popularity?
- Ratings Distribution since 1960
- What are the top rated movies?
Data and Setup¶
Import visualization libraries and set %matplotlib inline and Importing numpy and pandas¶
In [8]:
import pandas as pd
import numpy as np
import operator
import csv
import seaborn as sns
from datetime import datetime
import matplotlib.pyplot as plt
%matplotlib inline
Read in the csv file as a DataFrame called df_movies from TMDb_Movies Dataset.¶
In [9]:
#loading the csv file
df_movies = pd.read_csv('tmdb-movies.csv')
2 Data Wrangling¶
Check the head of df_movies from TMDb_Movies Dataset.¶
In [10]:
# first of all, I have to print the first 5 rows and define columns from TMDb_Movies Dataset.
df_movies.head()
Out[10]:
Checking the Dataset of df_movies, by using the function '.shape'¶
In [11]:
df_movies.shape
Out[11]:
Here we can see that we have 10866 rows and 21 columns
Checking the tail of df_movies from TMDb_Movies Dataset, using the function 'tail()'¶
In [12]:
df_movies.tail()
Out[12]:
I need to check the contents of my database from TMDb_Movies Dataset, using the info() function¶
In [13]:
df_movies.info()
Checking type from TMDb_Movies Dataset,¶
In [28]:
df_movies.dtypes
Out[28]:
3. Exploratory Data Analysis¶
Data Visualization¶
To create a histogram, I used pandas hist() method. Calling the hist() method on a pandas dataframe will return histograms for all non-nuisance series in the dataframe.
In [5]:
df_movies.hist(figsize=(20,20));
In [16]:
df_movies.describe()
Out[16]:
Check the columns from TMDb_Movies Dataset.¶
In [23]:
df_movies.columns
Out[23]:
Q1 Data Cleaning - Filling zero Values¶
Looking at the data above, budget, revenue and runtime all contain values of 0. Fill these in with the average of each column.
In [29]:
# Get average of budget_adj
print(df_movies['budget'].mean())
In [30]:
# Get average of revenue
print(df_movies['revenue'].mean())
In [31]:
# Get average of runtime
print(df_movies['runtime'].mean())
In [32]:
# Replace 0 values with mean
df_movies['budget'] = df_movies['budget'].replace(0, 14625701.09414688)
In [33]:
# Replace 0 values with mean
df_movies['revenue'] = df_movies['revenue'].replace(0, 39823319.79339223)
In [34]:
# Replace 0 values with mean
df_movies['runtime'] = df_movies['runtime'].replace(0, 102.07086324314375)
df_movies.describe()
Out[34]:
After checking our new data, we can clearly notice the boxes with the mention 0 have been replaced by the average of budget, revenue and runtime.
In [41]:
# Find out if there are any duplicate rows
sum(df_movies.duplicated())
Out[41]:
In [42]:
# Remove the duplicated rows
df_movies.drop_duplicates(inplace=True)
Q2 what is the average budget¶
To find the budget average from TMDb_Movies Dataset, by using Mean method¶
In [38]:
df_movies.budget.mean()
Out[38]:
What are the top 5 Budget from TMDb_Movies Dataset?¶
In [39]:
df_movies['budget'].value_counts().head(5)
Out[39]:
Histogram for budget¶
In [43]:
df_movies.budget.plot(kind = 'hist', color = 'red', bins = 25)
Out[43]:
In [34]:
# plot relationship between budget and runtime output
df_movies.plot(x='budget', y='runtime', kind='scatter');
In [35]:
# plot relationship between budget and revenue output
df_movies.plot(x='budget', y='revenue', kind='scatter');
In [36]:
# plot relationship between budget and release_year output
df_movies.plot(x='budget', y='release_year', kind='scatter');
Take a look at the 'budget' column, how many unique title are from TMDb_Movies Dataset?¶
In [39]:
df_movies['budget'].nunique()
Out[39]:
Q3 Creating new features¶
Using .apply() with a custom lambda expression to create a new column called "date" that contains this string value.
In [46]:
df_movies['release_date'].iloc[0]
Out[46]:
In [47]:
df_movies['date'] = df_movies['release_date'].apply(lambda original_title: original_title.split(':')[0])
In [48]:
df_movies['date'].value_counts(10)
Out[48]:
In [7]:
df_movies['popularity'].unique()
Out[7]:
In [5]:
Out[5]:
Now use seaborn to create a countplot from TMDb_Movies Dataset.¶
Checking Vote_averange by using plot¶
In [56]:
df_movies.vote_average.plot(kind = 'hist', color = 'red', bins = 25)
Out[56]:
let us visualize also boxplot as kind¶
In [57]:
x = df_movies.vote_average.plot(kind = 'box')
Q4 Ratings Distribution since 1960¶
The following code creates a graph showing the distribution of ratings by year.¶
In [58]:
sns.set(rc={'figure.figsize':(15,15)}, font_scale=1.3)
temp_df = df_movies[["release_year", "vote_average"]]
sns.set_style("whitegrid")
ax = sns.violinplot(x = temp_df.vote_average, y = temp_df.release_year, orient ="h")
ax.set(xlabel='movie ratings distributions', ylabel='years', title = 'movie ratings distributions per year')
plt.show()
release_year in percent¶
In [55]:
df = pd.DataFrame(dict(x=np.random.poisson(4, 500)))
ax = sns.barplot(x="release_year", y="release_year", data=df_movies, estimator=lambda x: len(x) / len(df_movies) * 100)
ax.set(ylabel="Percent")
Out[55]:
Q5 What are the variables that affect a film's revenue and popularity?¶
In [59]:
df_movies.corr()
Out[59]:
We will now examine one of each level of correlation.¶
In [61]:
# Plotting scatterplots to view correlation visually
sns.regplot(x = df_movies['budget'], y = df_movies['revenue'], fit_reg = False)
# Obtaining plot size
fig_size = plt.rcParams["figure.figsize"]
# Changing the length and width of the plot
fig_size[0] = 12
fig_size[1] = 8
plt.rcParams["figure.figsize"] = fig_size
plt.title('Budget vs Revenue', fontsize = 18)
plt.xlabel('Budget', fontsize = 16)
plt.ylabel('Revenue', fontsize = 16);
In [ ]:
In [69]:
#function which will take any column as argument from and keep its track
def calculate_count(column):
# Convert column to string and seperate it by '|'
data = df_movies[column].str.cat(sep = '|')
# Storing the values seperately in a Pandas series
data = pd.Series(data.split('|'))
count = data.value_counts(ascending = False)
return count
In [70]:
#variable to store the retured value#variabl
count = calculate_count('genres')
#printing top 5 values
count.head()
Out[70]:
In [71]:
count.plot(kind='pie', figsize = (12, 12));
4. Conclusions¶
In conclusion, after observation and analysis of the data, I would say that the world of cinema has evolved considerably. From 1960 to 2015, the gap is really considerable, in terms of allocated budget, popularity and others. In terms of percentage, I would say the cement has reached the top, with the budgets out of the norm and the popularity. Talking about the genre of film, according to my analysis, films in the Drama genre have more popularity, followed by Comedy, Thriller, Action and then Romance, which are the top 5.
In [ ]: