What is a Library?
Essentially a list of pre-written code that you can use to streamline and clean up your program.
Libraries can help simplify complex programs
APIS are specifications for how the procedures in a library behave, and how they can be used
Documentations for an API/library is necessary in understanding the behaviors provided by the API/library and how to use them
Libraries that we will go over: Requests, Pillow, Pandas, Numpy, Scikit-Learn, TensorFlow, matplotlib.
Required Installations
Please run the following commands in your vscode terminal in order to continue the lesson
- pip install numpy
- pip install matplotlib
- pip install scikit-learn
- pip install pillow
- pip install pandas
- pip install tensorflow
- pip install requests
Images using requests and pillow libraries
‘Requests’ is focused on handling HTTP requests and web data while ‘Pillow’ is designed for data manipulation and analysis It’s common to see them used together in data-related assignments where data is fetched by HTTP requests using Requests and then processed and analyzed with Pandas.
Here’s an example:
import requests
from PIL import Image
from io import BytesIO
# Step 1: Download an image using Requests
image_url = "https://example.com/path/to/your/image.jpg" # Replace with the actual URL of the image you want to download
response = requests.get(image_url)
if response.status_code == 200:
# Step 2: Process the downloaded image using Pillow
image_data = BytesIO(response.content) # Create an in-memory binary stream from the response content
img = Image.open(image_data) # Open the image using Pillow
# Perform image processing tasks here, like resizing or applying filters
img = img.resize((x, y)) # Resize the image and replace x,y with desired amounts
# Step 3: Save the processed image using Pillow
img.save("processed_image.jpg") # Save the processed image to a file
print("Image downloaded, processed, and saved.")
else:
print(f"Failed to download image. Status code: {response.status_code}")
In this code, we use the Requests library to download an image from a URL and then if the download is successful the HTTP status code 200 will pop up, and from there we create an in-memory binary stream (BytesIO) from the response content. We then use the Pillow library to open the image, make any necessary changes, and save the processed image to a file.
Here’s a step by step tutorial on how we wrote this code: 1)We started by importing the necessary libraries, which were Requests, Pillow, and io.
2)Download the Image
3)Use the Requests library to send an HTTP GET request to the URL to download the image. Check the response status code to make sure the download goes through(status code 200).
4)If the download is successful, create an in-memory binary stream (BytesIO) from the response content. Process the Image:
5)Utilize the Pillow library to open the image from the binary stream. Change photo to desired preference(ie: size) Save the Processed Image:
6)Save the processed image to a file using Pillow. Choose a filename and file format for the saved image.
Hack 1
Write a Python code that accomplishes the following tasks:
Downloads an image from a specified URL using the Requests library. Processes the downloaded image (like resizing) using the Pillow library. Save the processed image to a file.
#Code here
import requests
from PIL import Image
from io import BytesIO
# Step 1: Download an image using Requests
image_url = "https://imageio.forbes.com/blogs-images/forbestechcouncil/files/2019/01/canva-photo-editor-8-7.jpg?height=640&width=640&fit=bounds'" # Replace with the actual URL of the image you want to download
response = requests.get(image_url)
if response.status_code == 200:
# Step 2: Process the downloaded image using Pillow
image_data = BytesIO(response.content) # Create an in-memory binary stream from the response content
img = Image.open(image_data) # Open the image using Pillow
# Perform image processing tasks here, like resizing or applying filters
img = img.resize((300, 500)) # Resize the image and replace x,y with desired amounts
# Step 3: Save the processed image using Pillow
img.save("processed_image.jpg") # Save the processed image to a file
print("Image downloaded, processed, and saved.")
else:
print(f"Failed to download image. Status code: {response.status_code}")
Image downloaded, processed, and saved.
Math Operations With Python Libraries
Numpy(Numerical Python) is used for numerical and scientific computing. It provides tools for handling large sets of numbers, such as data tables and arrays. Numpy makes it easier and more efficient to do mathematical tasks.
The Matplotlib library lets you create a visual representation of your data (graphs, charts, and etc.)
Example Sine Graph
Uses numpy and matplotlib libaries
import numpy as np
import matplotlib.pyplot as plt
# Generate sample data with NumPy
x = np.linspace(0, 2 * np.pi, 100)
# Create an array of values from 0 to 2*pi
# 100 is included to have 100 points distributed between 0 and 2π to make graph smoother
y = np.sin(x)
# Compute the sine of each value
# Create a simple line plot using Matplotlib
plt.plot(x, y, label='Sine Function', color='blue', linestyle='-') # Create the plot
plt.title('Sine Function') # Set the title
plt.xlabel('x') # Label for the x-axis
plt.ylabel('sin(x)') # Label for the y-axis
plt.grid(True) # Display a grid
plt.legend() # Show the legend
plt.show() # Display the plot
Hack 2
Using the data from the numpy library, create a visual graph using different matplotlib functions.
import numpy as np
import matplotlib.pyplot as plt
# Generate data for two lines
x = np.linspace(0, 10, 50) # Create an array of values from 0 to 10
y1 = 2 * x + 1 # Set of data poits
# Create and display a plot using Matplotlib
# your code here
plt.plot(x, y1, label='Line Function', color='purple', linestyle='-') # Create the plot
plt.title('Line Function') # Set the title
plt.xlabel('x') # Label for the x-axis
plt.ylabel('y') # Label for the y-axis
plt.grid(True) # Display a grid
plt.legend() # Show the legend
plt.show() # Display the plot
Tensor Flow is used in deep learning and neural networks, while scikit-learn is used for typical machine learning tasks. When used together, they can tackle machine learning projects. In the code below, Tensor Flow is used for model creation and training. Scikit-learn is used for data-processing and model evaluation.
Pip install tensorflow scikit-learn
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import StandardScaler
from tensorflow import keras
from tensorflow.keras import layers
# Generate synthetic data
np.random.seed(0)
X = np.random.rand(100, 1) # Feature
y = 2 * X + 1 + 0.1 * np.random.randn(100, 1) # Target variable with noise
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Standardize the features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Create a simple linear regression model using TensorFlow and Keras
model = keras.Sequential([
layers.Input(shape=(1,)),
layers.Dense(1)
])
# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error')
# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=2)
# Make predictions on the test set
y_pred = model.predict(X_test)
# Calculate the Mean Squared Error on the test set
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse:.4f}")
Epoch 1/100
3/3 - 0s - loss: 4.7445 - 283ms/epoch - 94ms/step
Epoch 2/100
3/3 - 0s - loss: 4.7272 - 3ms/epoch - 1ms/step
Epoch 3/100
3/3 - 0s - loss: 4.7104 - 3ms/epoch - 982us/step
Epoch 4/100
3/3 - 0s - loss: 4.6929 - 3ms/epoch - 926us/step
Epoch 5/100
3/3 - 0s - loss: 4.6764 - 3ms/epoch - 1ms/step
Epoch 6/100
3/3 - 0s - loss: 4.6598 - 4ms/epoch - 1ms/step
Epoch 7/100
3/3 - 0s - loss: 4.6422 - 3ms/epoch - 977us/step
Epoch 8/100
3/3 - 0s - loss: 4.6253 - 3ms/epoch - 966us/step
Epoch 9/100
3/3 - 0s - loss: 4.6094 - 3ms/epoch - 924us/step
Epoch 10/100
3/3 - 0s - loss: 4.5918 - 3ms/epoch - 1ms/step
Epoch 11/100
3/3 - 0s - loss: 4.5750 - 4ms/epoch - 1ms/step
Epoch 12/100
3/3 - 0s - loss: 4.5581 - 3ms/epoch - 925us/step
Epoch 13/100
3/3 - 0s - loss: 4.5430 - 3ms/epoch - 952us/step
Epoch 14/100
3/3 - 0s - loss: 4.5253 - 4ms/epoch - 1ms/step
Epoch 15/100
3/3 - 0s - loss: 4.5090 - 3ms/epoch - 1ms/step
Epoch 16/100
3/3 - 0s - loss: 4.4924 - 3ms/epoch - 1ms/step
Epoch 17/100
3/3 - 0s - loss: 4.4767 - 5ms/epoch - 2ms/step
Epoch 18/100
3/3 - 0s - loss: 4.4594 - 4ms/epoch - 1ms/step
Epoch 19/100
3/3 - 0s - loss: 4.4427 - 4ms/epoch - 1ms/step
Epoch 20/100
3/3 - 0s - loss: 4.4272 - 3ms/epoch - 954us/step
Epoch 21/100
3/3 - 0s - loss: 4.4107 - 4ms/epoch - 1ms/step
Epoch 22/100
3/3 - 0s - loss: 4.3938 - 4ms/epoch - 1ms/step
Epoch 23/100
3/3 - 0s - loss: 4.3778 - 3ms/epoch - 1ms/step
Epoch 24/100
3/3 - 0s - loss: 4.3616 - 3ms/epoch - 939us/step
Epoch 25/100
3/3 - 0s - loss: 4.3458 - 4ms/epoch - 1ms/step
Epoch 26/100
3/3 - 0s - loss: 4.3296 - 4ms/epoch - 1ms/step
Epoch 27/100
3/3 - 0s - loss: 4.3131 - 3ms/epoch - 1ms/step
Epoch 28/100
3/3 - 0s - loss: 4.2981 - 3ms/epoch - 954us/step
Epoch 29/100
3/3 - 0s - loss: 4.2816 - 3ms/epoch - 1ms/step
Epoch 30/100
3/3 - 0s - loss: 4.2662 - 3ms/epoch - 975us/step
Epoch 31/100
3/3 - 0s - loss: 4.2514 - 3ms/epoch - 979us/step
Epoch 32/100
3/3 - 0s - loss: 4.2360 - 3ms/epoch - 955us/step
Epoch 33/100
3/3 - 0s - loss: 4.2196 - 3ms/epoch - 961us/step
Epoch 34/100
3/3 - 0s - loss: 4.2045 - 3ms/epoch - 1ms/step
Epoch 35/100
3/3 - 0s - loss: 4.1885 - 3ms/epoch - 924us/step
Epoch 36/100
3/3 - 0s - loss: 4.1730 - 3ms/epoch - 891us/step
Epoch 37/100
3/3 - 0s - loss: 4.1573 - 3ms/epoch - 973us/step
Epoch 38/100
3/3 - 0s - loss: 4.1415 - 3ms/epoch - 1ms/step
Epoch 39/100
3/3 - 0s - loss: 4.1265 - 3ms/epoch - 1ms/step
Epoch 40/100
3/3 - 0s - loss: 4.1109 - 3ms/epoch - 946us/step
Epoch 41/100
3/3 - 0s - loss: 4.0954 - 3ms/epoch - 908us/step
Epoch 42/100
3/3 - 0s - loss: 4.0793 - 3ms/epoch - 978us/step
Epoch 43/100
3/3 - 0s - loss: 4.0646 - 4ms/epoch - 1ms/step
Epoch 44/100
3/3 - 0s - loss: 4.0491 - 10ms/epoch - 3ms/step
Epoch 45/100
3/3 - 0s - loss: 4.0342 - 7ms/epoch - 2ms/step
Epoch 46/100
3/3 - 0s - loss: 4.0189 - 5ms/epoch - 2ms/step
Epoch 47/100
3/3 - 0s - loss: 4.0041 - 4ms/epoch - 1ms/step
Epoch 48/100
3/3 - 0s - loss: 3.9886 - 3ms/epoch - 1ms/step
Epoch 49/100
3/3 - 0s - loss: 3.9745 - 3ms/epoch - 954us/step
Epoch 50/100
3/3 - 0s - loss: 3.9597 - 3ms/epoch - 982us/step
Epoch 51/100
3/3 - 0s - loss: 3.9446 - 4ms/epoch - 1ms/step
Epoch 52/100
3/3 - 0s - loss: 3.9300 - 3ms/epoch - 1ms/step
Epoch 53/100
3/3 - 0s - loss: 3.9153 - 3ms/epoch - 1ms/step
Epoch 54/100
3/3 - 0s - loss: 3.9004 - 3ms/epoch - 1ms/step
Epoch 55/100
3/3 - 0s - loss: 3.8857 - 3ms/epoch - 1ms/step
Epoch 56/100
3/3 - 0s - loss: 3.8716 - 9ms/epoch - 3ms/step
Epoch 57/100
3/3 - 0s - loss: 3.8569 - 14ms/epoch - 5ms/step
Epoch 58/100
3/3 - 0s - loss: 3.8427 - 14ms/epoch - 5ms/step
Epoch 59/100
3/3 - 0s - loss: 3.8279 - 4ms/epoch - 1ms/step
Epoch 60/100
3/3 - 0s - loss: 3.8130 - 10ms/epoch - 3ms/step
Epoch 61/100
3/3 - 0s - loss: 3.7987 - 6ms/epoch - 2ms/step
Epoch 62/100
3/3 - 0s - loss: 3.7844 - 4ms/epoch - 1ms/step
Epoch 63/100
3/3 - 0s - loss: 3.7700 - 4ms/epoch - 1ms/step
Epoch 64/100
3/3 - 0s - loss: 3.7554 - 3ms/epoch - 993us/step
Epoch 65/100
3/3 - 0s - loss: 3.7407 - 4ms/epoch - 1ms/step
Epoch 66/100
3/3 - 0s - loss: 3.7271 - 3ms/epoch - 1ms/step
Epoch 67/100
3/3 - 0s - loss: 3.7123 - 3ms/epoch - 1ms/step
Epoch 68/100
3/3 - 0s - loss: 3.6980 - 3ms/epoch - 1ms/step
Epoch 69/100
3/3 - 0s - loss: 3.6838 - 4ms/epoch - 1ms/step
Epoch 70/100
3/3 - 0s - loss: 3.6699 - 3ms/epoch - 987us/step
Epoch 71/100
3/3 - 0s - loss: 3.6560 - 3ms/epoch - 970us/step
Epoch 72/100
3/3 - 0s - loss: 3.6416 - 3ms/epoch - 1ms/step
Epoch 73/100
3/3 - 0s - loss: 3.6277 - 4ms/epoch - 1ms/step
Epoch 74/100
3/3 - 0s - loss: 3.6139 - 3ms/epoch - 1ms/step
Epoch 75/100
3/3 - 0s - loss: 3.5999 - 3ms/epoch - 1ms/step
Epoch 76/100
3/3 - 0s - loss: 3.5868 - 6ms/epoch - 2ms/step
Epoch 77/100
3/3 - 0s - loss: 3.5726 - 6ms/epoch - 2ms/step
Epoch 78/100
3/3 - 0s - loss: 3.5584 - 10ms/epoch - 3ms/step
Epoch 79/100
3/3 - 0s - loss: 3.5454 - 4ms/epoch - 1ms/step
Epoch 80/100
3/3 - 0s - loss: 3.5318 - 4ms/epoch - 1ms/step
Epoch 81/100
3/3 - 0s - loss: 3.5182 - 4ms/epoch - 1ms/step
Epoch 82/100
3/3 - 0s - loss: 3.5050 - 4ms/epoch - 1ms/step
Epoch 83/100
3/3 - 0s - loss: 3.4912 - 4ms/epoch - 1ms/step
Epoch 84/100
3/3 - 0s - loss: 3.4781 - 4ms/epoch - 1ms/step
Epoch 85/100
3/3 - 0s - loss: 3.4645 - 3ms/epoch - 1ms/step
Epoch 86/100
3/3 - 0s - loss: 3.4511 - 4ms/epoch - 1ms/step
Epoch 87/100
3/3 - 0s - loss: 3.4376 - 4ms/epoch - 1ms/step
Epoch 88/100
3/3 - 0s - loss: 3.4253 - 4ms/epoch - 1ms/step
Epoch 89/100
3/3 - 0s - loss: 3.4110 - 4ms/epoch - 1ms/step
Epoch 90/100
3/3 - 0s - loss: 3.3980 - 4ms/epoch - 1ms/step
Epoch 91/100
3/3 - 0s - loss: 3.3848 - 4ms/epoch - 1ms/step
Epoch 92/100
3/3 - 0s - loss: 3.3718 - 4ms/epoch - 1ms/step
Epoch 93/100
3/3 - 0s - loss: 3.3587 - 4ms/epoch - 1ms/step
Epoch 94/100
3/3 - 0s - loss: 3.3450 - 4ms/epoch - 1ms/step
Epoch 95/100
3/3 - 0s - loss: 3.3325 - 4ms/epoch - 1ms/step
Epoch 96/100
3/3 - 0s - loss: 3.3192 - 3ms/epoch - 1ms/step
Epoch 97/100
3/3 - 0s - loss: 3.3063 - 4ms/epoch - 1ms/step
Epoch 98/100
3/3 - 0s - loss: 3.2930 - 4ms/epoch - 1ms/step
Epoch 99/100
3/3 - 0s - loss: 3.2801 - 3ms/epoch - 1ms/step
Epoch 100/100
3/3 - 0s - loss: 3.2673 - 4ms/epoch - 1ms/step
1/1 [==============================] - 0s 74ms/step
Mean Squared Error: 2.4938
A decrease in loss and time metrics (ms/epoch and ms/step) shows the efficiency increases as the training epochs increases
Hack
fill in the missing code to match the custom data set
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import StandardScaler
from tensorflow import keras
from tensorflow.keras import layers
# Generate a custom dataset (replace this with your data loading code)
np.random.seed(0)
X = np.random.rand(100, 1) # Feature
y = 2 * X + 1 + 0.1 * np.random.randn(100, 1)
# Synthetic data: House prices based on number of bedrooms and square footage
np.random.seed(0)
num_samples = 100
bedrooms = np.random.randint(1, 5, num_samples)
square_footage = np.random.randint(1000, 2500, num_samples)
house_prices = 100000 + 50000 * bedrooms + 100 * square_footage + 10000 * np.random.randn(num_samples)
# Combine features (bedrooms and square footage) into one array
X = np.column_stack((bedrooms, square_footage))
y = house_prices.reshape(-1, 1)
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Standardize the features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Create a regression model using TensorFlow and Keras
model = keras.Sequential([
layers.Input(shape=(1,)),
layers.Dense(1)
]),6
# Input shape adjusted to the number of features
# Output layer for regression
# Compile the model for regression
model.compile(optimizer='adam', loss='mean_squared_error')
# Using MSE as the loss function
# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=2)
# Make predictions on the test set
y_pred = model.predict(X_test)
# Calculate the Mean Squared Error on the test set
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse:.4f}")
Epoch 1/100
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[12], line 44
41 model.compile(optimizer='adam', loss='mean_squared_error')
42 # Using MSE as the loss function
43 # Train the model
---> 44 model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=2)
46 # Make predictions on the test set
47 y_pred = model.predict(X_test)
File /opt/homebrew/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
File /var/folders/h2/d0z2cz5x4w53jv31kmbt945c0000gp/T/__autograph_generated_filebqd3y9fu.py:15, in outer_factory.<locals>.inner_factory.<locals>.tf__train_function(iterator)
13 try:
14 do_return = True
---> 15 retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
16 except:
17 do_return = False
ValueError: in user code:
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/training.py", line 1377, in train_function *
return step_function(self, iterator)
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/training.py", line 1360, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/training.py", line 1349, in run_step **
outputs = model.train_step(data)
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/training.py", line 1126, in train_step
y_pred = self(x, training=True)
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/input_spec.py", line 280, in assert_input_compatibility
raise ValueError(
ValueError: Exception encountered when calling layer 'sequential_1' (type Sequential).
Input 0 of layer "dense_1" is incompatible with the layer: expected axis -1 of input shape to have value 1, but received input with shape (None, 2)
Call arguments received by layer 'sequential_1' (type Sequential):
• inputs=tf.Tensor(shape=(None, 2), dtype=float32)
• training=True
• mask=None
HOMEWORK 1
Create a GPA calculator using Pandas and Matplot libraries and make: 1) A dataframe 2) A specified dictionary 3) and a print function that outputs the final GPA
Extra points can be earned with creativity.
# your code here
import pandas as pd
import matplotlib.pyplot as plt
data = {
'Subject': ['AP Calculus', 'AP Chemistry', 'AP World', 'Honors Humanitites', 'Photography'],
'Credits': [5, 5, 5, 5, 5],
'Grade': ['A', 'B', 'B', 'A', 'A'],
}
df = pd.DataFrame(data)
grade_points = {
'A': 4.0,
'A-': 3.6,
'B+': 3.2,
'B': 3.0,
'B-': 2.6,
'C+': 2.3,
'C': 2.0,
'C-': 1.6,
'D+': 1.3,
'D': 1.0,
'F': 0.0,
}
df['Grade Points'] = df['Grade'].map(grade_points)
df['Weighted Points'] = df['Credits'] * df['Grade Points']
total_credits = df['Credits'].sum()
total_weighted_points = df['Weighted Points'].sum()
final_gpa = total_weighted_points / total_credits
def print_final_gpa():
print("Subject-wise GPA:")
print(df)
print("\nTotal Credits: ", total_credits)
print("Total Weighted Points: ", total_weighted_points)
print("Final GPA: {:.2f}".format(final_gpa))
print_final_gpa()
plt.bar(df['Subject'], df['Grade Points'], color='skyblue')
plt.xlabel('Subjects')
plt.ylabel('Grade Points')
plt.title('GPA Distribution')
plt.xticks(rotation=15)
plt.ylim(0, 4.0)
plt.show()
Subject-wise GPA:
Subject Credits Grade Grade Points Weighted Points
0 AP Calculus 5 A 4.0 20.0
1 AP Chemistry 5 B 3.0 15.0
2 AP World 5 B 3.0 15.0
3 Honors Humanitites 5 A 4.0 20.0
4 Photography 5 A 4.0 20.0
Total Credits: 25
Total Weighted Points: 90.0
Final GPA: 3.60
HOMEWORK 2
Import and use the “random” library to generate 50 different points from the range 0-100, then display the randomized data using a scatter plot.
Extra points can be earned with creativity.
# your code here
import random
import matplotlib.pyplot as plt
random_data = [random.randint(0, 100) for _ in range(50)]
x = list(range(1, 51))
y = random_data
plt.figure(figsize=(10, 6))
plt.scatter(x, y, c='b', marker='o', label='Random Data Points', s=50)
plt.xlabel('Data Point Index')
plt.ylabel('Value')
plt.title('Scatter Plot of 50 Random Data Points (0-100)')
plt.legend()
plt.grid(True)
plt.show()
