diff --git a/readme25-27.md b/readme25-27.md index c7b07fb..6cfa970 100644 --- a/readme25-27.md +++ b/readme25-27.md @@ -25,7 +25,14 @@ - [Creating DataFrams from list of dictionaries](#creating-dataframs-from-list-of-dictionaries) - [Reading CSV File using pandas](#reading-csv-file-using-pandas) - [Data Exploration](#data-exploration) - - [💻 Exercises: Day 25](#%f0%9f%92%bb-exercises-day-25) + - [Modifying DataFrame](#modifying-dataframe) + - [Create a DataFrame](#create-a-dataframe) + - [Adding new column](#adding-new-column) + - [Modifying column values](#modifying-column-values) + - [Formating DataFrame column](#formating-dataframe-column) + - [Checking data types of Column values](#checking-data-types-of-column-values) + - [Boolean Indexing](#boolean-indexing) + - [Exercises: Day 25](#exercises-day-25) - [📘 Day 26](#%f0%9f%93%98-day-26) - [Python for Web](#python-for-web) - [Flask](#flask) @@ -61,27 +68,26 @@ - [💻 Exercises: Day 27](#%f0%9f%92%bb-exercises-day-27) # 📘 Day 25 - ## Pandas Pandas is an open source,high-performance, easy-to-use data structures and data analysis tools for the Python programming language. Pandas adds data structures and tools designed to work with table-like data which is Series and Data Frames Pandas provides tools for data manipulation: reshaping, merging, sorting, slicing, aggregation and imputation. - ```py pip install conda conda install pandas ``` +Pandas data structure is based on *Series* and *DataFrames* +A series is a column and a DataFrame is a multidimensional table made up of collection of series. In order to create a pandas series we should use numpy to create a one dimensional arrays or a python list. +Let's see an example of a series: -Pandas data structure is based on _Series_ and _DataFrames_ -A series is a column and a DataFrame is a multidimensional table made up of collection of series. In order to create a pandas series we should use numpy to create a one dimensional arrays. -An example of a series, names +Names pandas Series -![pandas series](images/pandas-series-1.png) +![pandas series](images/pandas-series-1.png) Countries Series -![pandas series](images/pandas-series-2.png) +![pandas series](images/pandas-series-2.png) Cities Series @@ -95,26 +101,31 @@ Let's see, an example of a pandas data frame: Data from is a collection of rows and columns. Look at the table below it has many columns than the above + ![Pandas data frame](images/pandas-dataframe-2.png) +Next, we will see how to import pandas and how to create Series and DataFrames using pandas + ## Importing pandas + ```python -import pandas as pd -import numpy as np +import pandas as pd # importing pandas as pd +import numpy as np # importing numpy as np ``` ### Creating Pandas Series with default index -```python -nums = [1, 2, 3,4,5] -s = pd.Series(nums) -``` ```python +nums = [1, 2, 3, 4,5] +s = pd.Series(nums) s ``` + + + 0 1 1 2 2 3 @@ -122,66 +133,99 @@ s 4 5 dtype: int64 -### Creating Pandas Series with custom index + + +### Creating Pandas Series with custom index + + +```python +nums = [1, 2, 3, 4, 5] +s = pd.Series(nums, index=[1, 2, 3, 4, 5]) +s + +``` + + + + + 1 1 + 2 2 + 3 3 + 4 4 + 5 5 + dtype: int64 + + + ```python fruits = ['Orange','Banana','Mangao'] -fruits = pd.Series(fruits, index=[1, 2,3]) -``` - -```python +fruits = pd.Series(fruits, index=[1, 2, 3]) fruits ``` + + + 1 Orange 2 Banana 3 Mangao dtype: object + + ### Creating Pandas Series from a dictionary + ```python dct = {'name':'Asabeneh','country':'Finland','city':'Helsinki'} ``` -```python -s = pd.Series(dct) -``` ```python +s = pd.Series(dct) s ``` + + + name Asabeneh country Finland city Helsinki dtype: object + + ### Creating a constant pandas series + ```python s = pd.Series(10, index = [1, 2,3]) -``` - -```python s ``` + + + 1 10 2 10 3 10 dtype: int64 -### Creating a pandas series using linspace - -```python -s = pd.Series(np.linspace(5, 20, 10)) -``` + + +### Creating a pandas series using linspace + ```python +s = pd.Series(np.linspace(5, 20, 10)) # linspace(starting, end, items) s ``` + + + 0 5.000000 1 6.666667 2 8.333333 @@ -194,23 +238,29 @@ s 9 20.000000 dtype: float64 + + ## DataFrames Pandas data frames can be created in different ways. ### Creating DataFrames from list of lists + ```python -data = [["Asabeneh", "Finland", "Helsink"], [ - "David", "UK", "London"], ["John", "Sweden", "Stockholm"]] +data = [ + ['Asabeneh', 'Finland', 'Helsink'], + ['David', 'UK', 'London'], + ['John', 'Sweden', 'Stockholm'] +] df = pd.DataFrame(data, columns=['Names','Country','City']) - -``` - -```python df + ``` + + +
@@ -258,18 +307,21 @@ df
+ + ### Creating DataFrame using Dictionary -```python -data = {"Name": ["Asabeneh", "David", "John"], "Country":[ - "Finland", "UK", "Sweden"], "City": ["Helsiki", "London", "Stockholm"]} -df = pd.DataFrame(data) -``` ```python +data = {'Name': ['Asabeneh', 'David', 'John'], 'Country':[ + 'Finland', 'UK', 'Sweden'], 'City': ['Helsiki', 'London', 'Stockholm']} +df = pd.DataFrame(data) df ``` + + +
@@ -317,20 +368,28 @@ df
+ + + +```python + +``` + ### Creating DataFrams from list of dictionaries + ```python data = [ - {"Name": "Asabeneh", "Country":"Finland","City":"Helsinki"}, - {"Name": "David", "Country":"UK","City":"London"}, - {"Name": "John", "Country":"Sweden","City":"Stockholm"}] + {'Name': 'Asabeneh', 'Country': 'Finland', 'City': 'Helsinki'}, + {'Name': 'David', 'Country': 'UK', 'City': 'London'}, + {'Name': 'John', 'Country': 'Sweden', 'City': 'Stockholm'}] df = pd.DataFrame(data) -``` - -```python df ``` + + +
@@ -378,8 +436,11 @@ df
+ + ## Reading CSV File using pandas + ```python import pandas as pd @@ -387,13 +448,16 @@ df = pd.read_csv('./data/weight-height.csv') ``` ### Data Exploration - Let's read only the first 5 rows using head() + ```python df.head() # give five rows we can increase the number of rows by passing argument to the head() method ``` + + +
@@ -453,28 +516,47 @@ df.head() # give five rows we can increase the number of rows by passing argumen
+ + As you can see the csv file has three rows:Gender, Height and Weight. But we don't know the number of rows. Let's use shape meathod. + ```python df.shape # as you can see 10000 rows and three columns ``` + + + (10000, 3) + + Let's get all the columns using columns. + + ```python df.columns ``` + + + Index(['Gender', 'Height', 'Weight'], dtype='object') + + Let's read only the last 5 rows using tail() + ```python df.tail() # tails give the last five rows, we can increase the rows by passing argument to tail method ``` + + +
@@ -534,22 +615,30 @@ df.tail() # tails give the last five rows, we can increase the rows by passing a
+ + Now, lets get specif colums using the column key + + ```python heights = df['Height'] # this is now a a series ``` + ```python heights ``` + + + 0 73.847017 1 68.781904 2 74.110105 3 71.730978 4 69.881796 - ... + ... 9995 66.172652 9996 67.067155 9997 63.867992 @@ -557,20 +646,27 @@ heights 9999 61.944246 Name: Height, Length: 10000, dtype: float64 + + + ```python weights = df['Weight'] # this is now a series ``` + ```python weights ``` + + + 0 241.893563 1 162.310473 2 212.740856 3 220.042470 4 206.349801 - ... + ... 9995 136.777454 9996 170.867906 9997 128.475319 @@ -578,16 +674,28 @@ weights 9999 113.649103 Name: Weight, Length: 10000, dtype: float64 + + + ```python len(heights) == len(weights) ``` + + + True + + + ```python heights.describe() # give statisical information about height data ``` + + + count 10000.000000 mean 66.367560 std 3.847528 @@ -598,10 +706,16 @@ heights.describe() # give statisical information about height data max 78.998742 Name: Height, dtype: float64 + + + ```python weights.describe() ``` + + + count 10000.000000 mean 161.440357 std 32.108439 @@ -612,10 +726,16 @@ weights.describe() max 269.989699 Name: Weight, dtype: float64 + + + ```python df.describe() # describe can also give statistical information from a datafrom ``` + + +
@@ -684,17 +803,845 @@ df.describe() # describe can also give statistical information from a datafrom
-## 💻 Exercises: Day 25 -1. Read the hacker_ness.csv file from data directory + +## Modifying DataFrame + + + +Modifying a DataFrame + * We can create a new DataFrame + * We can create a new column and add to DataFrame, + * we can remove an existing column from DataFrame, + * we can modify an existing column from DataFrame, + * we can change the data type of column values from DataFrame + +### Create a DataFrame +All the time, first we import the necessary packages. Now, lets import pandas and numpy two best friends ever. + + +```python +import pandas as pd +import numpy as np +data = [ + {"Name": "Asabeneh", "Country":"Finland","City":"Helsinki"}, + {"Name": "David", "Country":"UK","City":"London"}, + {"Name": "John", "Country":"Sweden","City":"Stockholm"}] +df = pd.DataFrame(data) +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCity
0AsabenehFinlandHelsinki
1DavidUKLondon
2JohnSwedenStockholm
+
+ + + +Adding column in DataFrame is like adding a key in dictionary. + +First let's use the previous example to create a DataFrame. After we create the DataFrame, we will start modifying the columns and column values. + +### Adding new column +Let's add a weight column in the DataFrame + + +```python +weights = [74, 78, 69] +df['Weight'] = weights +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeight
0AsabenehFinlandHelsinki74
1DavidUKLondon78
2JohnSwedenStockholm69
+
+ + + +Let's add a height column in the DataFrame + + +```python +heights = [173, 175, 169] +df['Height'] =heights +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeight
0AsabenehFinlandHelsinki74173
1DavidUKLondon78175
2JohnSwedenStockholm69169
+
+ + + +As you can see from the above DataFrame, now we new added columns, the Weight and Height. Let's add one additional column by called BMI(Body Mass Index) by calculating their BMI using thier mass and height. BMI is mass divided by height square meter(Weight/Height * Height). + +As you can see, the hieght is in centimeter, so we shoud change the height to meter. So, let's modify the height row + +### Modifying column values + + +```python +df['Height'] = df['Height'] * 0.01 +df + +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeight
0AsabenehFinlandHelsinki741.73
1DavidUKLondon781.75
2JohnSwedenStockholm691.69
+
+ + + + +```python +# Using function makes our code clean but you can just calculate the bmi without function +def calculate_bmi (): + weights = df['Weight'] + heights = df['Height'] + bmi = [] + for w,h in zip(weights, heights): + b = w/(h*h) + bmi.append(b) + return bmi + +bmi = calculate_bmi() + +``` + + +```python +df['BMI'] = bmi +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMI
0AsabenehFinlandHelsinki741.7324.725183
1DavidUKLondon781.7525.469388
2JohnSwedenStockholm691.6924.158818
+
+ + + +### Formating DataFrame column + +The BMI of the above DataFrame has is float with many significant digits after decimal. Let's make it to have only one significant digit after point. + + +```python +df['BMI'] = round(df['BMI'], 1) +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMI
0AsabenehFinlandHelsinki741.7324.7
1DavidUKLondon781.7525.5
2JohnSwedenStockholm691.6924.2
+
+ + + +The information in the DataFrame seems not yet complete, let's add birth year and current year columns. + + +```python +birth_year = ['1769', '1985', '1990'] +current_year = pd.Series(2019, index=[0, 1,2]) +df['Birth Year'] = birth_year +df['Current Year'] = current_year +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMIBirth YearCurrent Year
0AsabenehFinlandHelsinki741.7324.717692019
1DavidUKLondon781.7525.519852019
2JohnSwedenStockholm691.6924.219902019
+
+ + + +## Checking data types of Column values + + +```python +df.Weight.dtype +``` + + + + + dtype('int64') + + + + +```python +df['Birth Year'].dtype # it give string object , we should change this to number + +``` + + + + + dtype('O') + + + + +```python +df['Birth Year'] = df['Birth Year'].astype('int') +df['Birth Year'].dtype # let's check the data type now +``` + + + + + dtype('int64') + + + + +```python +df['Current Year'] = df['Current Year'].astype('int') +df['Current Year'].dtype +``` + + + + + dtype('int64') + + + +Now, the column values of birth year and current year are integers. We can calculate the age. + + +```python +ages = df['Current Year'] - df['Birth Year'] +ages +``` + + + + + 0 250 + 1 34 + 2 29 + dtype: int64 + + + + +```python +df['Ages'] = ages +df +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMIBirth YearCurrent YearAges
0AsabenehFinlandHelsinki741.7324.717692019250
1DavidUKLondon781.7525.51985201934
2JohnSwedenStockholm691.6924.21990201929
+
+ + + +The person in the first row lives 250 years. It is unlikely for someone to live 250 years. Either it is a typo or the data is cooked. So lets fill that data with average of the columns without including outlier. + +mean = (34 + 29)/ 2 + + +```python +mean = (34 + 29)/ 2 +mean +``` + + + + + 31.5 + + + +### Boolean Indexing + + +```python +df[df['Ages'] > 120] +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMIBirth YearCurrent YearAges
0AsabenehFinlandHelsinki741.7324.717692019250
+
+ + + + +```python +df[df['Ages'] < 120] +``` + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCountryCityWeightHeightBMIBirth YearCurrent YearAges
1DavidUKLondon781.7525.51985201934
2JohnSwedenStockholm691.6924.21990201929
+
+ + + + +```python +df['Ages'] = df[df['Ages'] > 120] + + +``` + +## Exercises: Day 25 +1. Read the hacker_ness.csv file from data directory 1. Get the first five rows 1. Get the last five rows 1. Get the title column as pandas series 1. Count the number of rows and columns - - Filter the titles which contain python - - Filter the titles which contain JavaScript - - Explore the data and make sense of the data - + * Filter the titles which contain python + * Filter the titles which contain JavaScript + * Explore the data and make sense of the data # 📘 Day 26 ## Python for Web @@ -1822,5 +2769,3 @@ Now, we have deleted the students collection from the database. --- -asabeneh@Asabeneh:~/Desktop/python_for_web\$ -``` diff --git a/readme28-30 b/readme28-30 new file mode 100644 index 0000000..ffa872a --- /dev/null +++ b/readme28-30 @@ -0,0 +1,8 @@ + +# 📘 Day 28 +## 💻 Exercises: Day 28 + + +[<< Part 8 ](https://github.com/Asabeneh/30-Days-Of-Python/blob/master/readme22-24.md) | [Part 10 >>](https://github.com/Asabeneh/30-Days-Of-Python/blob/master/readme28-30.md) + +--- \ No newline at end of file