1. Implement the KNN Algorithm on the iris dataset and obtain the desired output.
CODE:
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn import datasets
iris=datasets.load_iris()
print("Iris Data set loaded...")
X_train, X_test, y_train, y_test = train_test_split(iris.data,iris.target,test_size=0.1)
print("Dataset is split into training and testing...")
print("Size of training data and its label",X_train.shape,y_train.shape)
print("Size of test data and its label",X_test.shape,y_test.shape)
for i in range(len(iris.target_names)):
print("label",i,"-",str(iris.target_names[i]))
classifier = KNeighborsClassifier(n_neighbors=1)
classifier.fit(X_train, y_train)
y_pred=classifier.predict(X_test)
print("Results of classification using KNN with k=1")
for r in range(0,len(X_test)):
print("Sample: ",str(X_test[r]),"Actual label: ",str(y_test[r]),"Predicted label: ",str(y_pred[r]))
print("Classification Accuracy: ",classifier.score(X_test,y_test))
from sklearn.metrics import classification_report,confusion_matrix
print('Confusion Matrix: ')
print(confusion_matrix(y_test,y_pred))
print('Accuracy Metrics: ')
print(classification_report(y_test,y_pred)
Output:
Iris Data set loaded...
Dataset is split into training and testing...
Size of training data and its label (135, 4) (135,)
Size of test data and its label (15, 4) (15,)
label 0 - setosa
label 1 - versicolor
label 2 - virginica
Results of classification using KNN with k=1
Sample: [5.2 4.1 1.5 0.1] Actual label: 0 Predicted label: 0
Sample: [6.5 3. 5.5 1.8] Actual label: 2 Predicted label: 2
Sample: [5.2 3.5 1.5 0.2] Actual label: 0 Predicted label: 0
Sample: [7.1 3. 5.9 2.1] Actual label: 2 Predicted label: 2
Sample: [5.2 2.7 3.9 1.4] Actual label: 1 Predicted label: 1
Sample: [6.3 2.3 4.4 1.3] Actual label: 1 Predicted label: 1
Sample: [6.4 3.2 4.5 1.5] Actual label: 1 Predicted label: 1
Sample: [6.9 3.1 5.4 2.1] Actual label: 2 Predicted label: 2
Sample: [6. 2.2 4. 1. ] Actual label: 1 Predicted label: 1
Sample: [6.4 3.1 5.5 1.8] Actual label: 2 Predicted label: 2
Sample: [6.6 2.9 4.6 1.3] Actual label: 1 Predicted label: 1
Sample: [5.5 3.5 1.3 0.2] Actual label: 0 Predicted label: 0
Sample: [6.7 2.5 5.8 1.8] Actual label: 2 Predicted label: 2
Sample: [7. 3.2 4.7 1.4] Actual label: 1 Predicted label: 1
Sample: [5.5 2.3 4. 1.3] Actual label: 1 Predicted label: 1
Classification Accuracy: 1.0
Confusion Matrix:
[[3 0 0]
[0 7 0]
[0 0 5]]
Accuracy Metrics:
precision recall f1-score support
0 1.00 1.00 1.00 3
1 1.00 1.00 1.00 7
2 1.00 1.00 1.00 5
accuracy 1.00 15
macro avg 1.00 1.00 1.00 15
weighted avg 1.00 1.00 1.00 1
Another implementation of KNN Algorithm
from sklearn.neighbors import KNeighborsClassifier
#create list of data points
data_points=[(2,10),(2, 6),(11,11), (6, 9), (6, 5), (1, 2), (5, 10), (4, 9),(10, 12),(7, 5),(9, 11),(4, 6), (3, 10), (3, 8),(6, 11)]
#create list of class labels
class_labels=["C2","C1","C3", "C2","C1","C1","C2","C2","C3","C1","C3","C1","C2","C2","C2"]
#create untrined model
untrained_model=KNeighborsClassifier(n_neighbors=3, metric="euclidean")
#train model using fit method
trained_model=untrained_model.fit(data_points,class_labels)
#predict class for model
predicted_class=trained_model.predict([(5,7)])
print("The data points are:")
print(data_points)
print("The class labels are:")
print(class_labels)
print("The predicted class label for (5,7) is:")
print(predicted_class)
OUTPUT:
The data points are:
[(2, 10), (2, 6), (11, 11), (6, 9), (6, 5), (1, 2), (5, 10), (4, 9), (10, 12), (7, 5), (9, 11), (4, 6), (3, 10), (3, 8), (6, 11)]
The class labels are:
['C2', 'C1', 'C3', 'C2', 'C1', 'C1', 'C2', 'C2', 'C3', 'C1', 'C3', 'C1', 'C2', 'C2', 'C2']
The predicted class label for (5,7) is:
['C1']
2. Implement Naïve Bayes algorithm on the iris dataset and display the accuracy and confusion matrix.
CODE:
from sklearn.naive_bayes import GaussianNB
from sklearn.datasets import load_iris
from sklearn import metrics
from sklearn.model_selection import train_test_split as split
dataset=load_iris()
x=dataset.data
y=dataset.target
x_train,x_test,y_train,y_test=split(x,y,test_size=0.2,random_state=1)
gnb=GaussianNB()
classifier=gnb.fit(x_train,y_train)
y_pred=classifier.predict(x_test)
print("Accuracy Metrics :")
print(metrics.classification_report(y_test,y_pred))
print("The accuracy of metrics is: ",metrics.accuracy_score(y_test,y_pred))
print("Confusion matrix: ")
print(metrics.confusion_matrix(y_test,y_pred))
Output:
Accuracy Metrics :
precision recall f1-score support
0 1.00 1.00 1.00 11
1 1.00 0.92 0.96 13
2 0.86 1.00 0.92 6
accuracy 0.97 30
macro avg 0.95 0.97 0.96 30
weighted avg 0.97 0.97 0.97 30
The accuracy of metrics is: 0.9666666666666667
Confusion matrix
[[11 0 0]
[ 0 12 1]
[ 0 0 6]]
3. Implement SVM program on the random dataset and plot a graph.
CODE:
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor
import numpy as np
x=np.linspace(0,10,100)
y_linear=2*x+3+np.random.randn(100)*0.5
y_complex=np.sin(x)+np.random.randn(100)*0.2
models=[LinearRegression(),DecisionTreeRegressor(max_depth=5)]
for model in models:
model.fit(x.reshape(-1,1),y_linear)
y_pred_linear=model.predict(x.reshape(-1,1))
model.fit(x.reshape(-1,1),y_complex)
y_pred_complex=model.predict(x.reshape(-1,1))
plt.figure(figsize=(6,2))
plt.plot(x,y_linear,label='True data (Linear)')
plt.plot(x,y_pred_linear,label='Predicted')
Output:
4. Implement SVM program on the iris dataset.
CODE:
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score,confusion_matrix
iris=datasets.load_iris()
xtrain,xtest,ytrain,ytest=train_test_split(iris.data[:,:2],iris.target,test_size=0.3,random_state=42)
scaler=StandardScaler()
xtrain=scaler.fit_transform(xtrain)
xtest=scaler.transform(xtest)
model=SVC(kernel='linear',C=2)
model.fit(xtrain,ytrain)
ypred=model.predict(xtest)
print(f"Accuracy: {accuracy_score(ytest,ypred)*100:.2f}%")
print(confusion_matrix(ytest,ypred))
Output:
Accuracy: 80.00%
[[19 0 0]
[ 0 7 6]
[ 0 3 10]]
5. Implement a Python program to demonstrate the working of the decision tree-based ID3 algorithm. Use an appropriate data set for building the decision tree.
CODE:
# Import necessary libraries
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, classification_report
# Load the Iris dataset iris = load_iris()
X = iris.data y = iris.target
# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# Initialize the Decision Tree Classifier with entropy criterion
clf = DecisionTreeClassifier(criterion='entropy')
# Train the classifier clf.fit(X_train, y_train)
# Predict the labels for the test set
y_pred = clf.predict(X_test)
# Calculate and print the accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy * 100:.2f}%")
# Print the classification report
print("\nClassification Report:")
print(classification_report(y_test, y_pred))
# Print the first 10 actual vs predicted labels
print("\nActual vs Predicted labels for the first 10 samples:")
for actual, predicted in zip(y_test[:10], y_pred[:10]): print(f"Actual: {actual}, Predicted: {predicted}")
Output:
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 19
1 1.00 1.00 1.00 13
2 1.00 1.00 1.00 13
accuracy 1.00 45
macro avg 1.00 1.00 1.00 45
weighted avg 1.00 1.00 1.00 45
Actual vs Predicted labels for the first 10 samples: Actual: 1, Predicted: 1
Actual: 0, Predicted: 0
Actual: 2, Predicted: 2
Actual: 1, Predicted: 1
Actual: 1, Predicted: 1
Actual: 0, Predicted: 0
Actual: 1, Predicted: 1
Actual: 2, Predicted: 2
Actual: 1, Predicted: 1
Actual: 1, Predicted: 1
6. Implement Random Forest-boosting algorithm using AdaBoost classifier.
CODE:
AdaBoost Classifier (Boosting)
from sklearn.ensemble import AdaBoostClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
# Load dataset
iris = load_iris()
X = iris.data
y = iris.target
# Split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42)
# Base learner (weak learner)
base_model = DecisionTreeClassifier(max_depth=1)
# AdaBoost model
ada_clf = AdaBoostClassifier(
estimator=base_model,
n_estimators=50,
learning_rate=1.0,
random_state=42
)
# Train
ada_clf.fit(X_train, y_train)
# Predict
y_pred_ada = ada_clf.predict(X_test)
# Evaluation
print("AdaBoost Accuracy:", accuracy_score(y_test, y_pred_ada))
print(classification_report(y_test, y_pred_ada, target_names=iris.target_names))
How AdaBoost Works
• Trains weak learners (small trees) sequentially
• Misclassified points get higher weight
• Next model focuses more on those errors
7. Implement Random Forest-boosting algorithm using Gradient Boosting classifier.
CODE:
Gradient Boosting Classifier
from sklearn.ensemble import GradientBoostingClassifier
# Gradient Boosting model
gb_clf = GradientBoostingClassifier(
n_estimators=100,
learning_rate=0.1,
max_depth=3,
random_state=42
)
# Train
gb_clf.fit(X_train, y_train)
# Predict
y_pred_gb = gb_clf.predict(X_test)
# Evaluation
print("Gradient Boosting Accuracy:", accuracy_score(y_test, y_pred_gb))
print(classification_report(y_test, y_pred_gb, target_names=iris.target_names))
How Gradient Boosting Works
• Builds trees sequentially
• Each new tree reduces previous errors
• Optimizes using gradient descent on loss function
(Optional) Random Forest for Comparison
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=100, random_state=42)
rf_clf.fit(X_train, y_train)
y_pred_rf = rf_clf.predict(X_test)
print("Random Forest Accuracy:", accuracy_score(y_test, y_pred_rf))
8. Develop a Python program for AND operation using perceptron.
CODE:
def activation(out,threshold):
if out>=threshold:
return 1
else:
return 0
def perceptron(and_input):
a=[0,0,1,1]
b=[0,1,0,1]
y=[0,0,0,1]
w=[1.4,1.5]
threshold=1
learning_rate=0.1
i=0
print("Perceptron Training : ")
print('######################')
print('------------')
while i<4:
summation = a[i]*w[0] + b[i]*w[1]
o= activation(summation,threshold)
print("Input : " + str(a[i]) + " , "+ str(b[i]))
print("Weights: " +str(w[0]) + " , " + str(w[1]))
print("summation : "+str(summation) + "threshold : "+str(threshold))
print("Actual Output : "+str(y[i])+" Predicted Output : "+str(o))
if(o!=y[i]):
print("____________\nUpdating weights")
w[0]=w[0]+learning_rate*(y[i]-o)*a[i]
print("w[o]",w[o])
w[1]=w[1]+learning_rate*(y[i]-o)*b[i]
print("Updated Weights : " + str(w[0]) + " , "+ str(w[1]))
i=-1
print("\nWeights Updated Training Again : ")
print("###################################")
i=i+1
print("---------------")
summation = and_input[0]*w[0] + and_input[1]*w[1]
return activation(summation,threshold)
and_input = [1,1]
print("AND Gate Output for " +str(and_input) + " : "+str(perceptron(and_input)))
Output:
Perceptron Training :
######################
------------
Input : 0 , 0
Weights: 1.4 , 1.5
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 1.5
summation : 1.5threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 1.5
Updated Weights : 1.4 , 1.4
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.4 , 1.4
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 1.4
summation : 1.4threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 1.4
Updated Weights : 1.4 , 1.2999999999999998
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.4 , 1.2999999999999998
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 1.2999999999999998
summation : 1.2999999999999998threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 1.2999999999999998
Updated Weights : 1.4 , 1.1999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.4 , 1.1999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 1.1999999999999997
summation : 1.1999999999999997threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 1.1999999999999997
Updated Weights : 1.4 , 1.0999999999999996
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.4 , 1.0999999999999996
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 1.0999999999999996
summation : 1.0999999999999996threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 1.0999999999999996
Updated Weights : 1.4 , 0.9999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.4 , 0.9999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.4 , 0.9999999999999997
summation : 0.9999999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 0
Weights: 1.4 , 0.9999999999999997
summation : 1.4threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 0.9999999999999997
Updated Weights : 1.2999999999999998 , 0.9999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.2999999999999998 , 0.9999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.2999999999999998 , 0.9999999999999997
summation : 0.9999999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 0
Weights: 1.2999999999999998 , 0.9999999999999997
summation : 1.2999999999999998threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 0.9999999999999997
Updated Weights : 1.1999999999999997 , 0.9999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.1999999999999997 , 0.9999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.1999999999999997 , 0.9999999999999997
summation : 0.9999999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 0
Weights: 1.1999999999999997 , 0.9999999999999997
summation : 1.1999999999999997threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 0.9999999999999997
Updated Weights : 1.0999999999999996 , 0.9999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 1.0999999999999996 , 0.9999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 1.0999999999999996 , 0.9999999999999997
summation : 0.9999999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 0
Weights: 1.0999999999999996 , 0.9999999999999997
summation : 1.0999999999999996threshold : 1
Actual Output : 0 Predicted Output : 1
____________
Updating weights
w[o] 0.9999999999999997
Updated Weights : 0.9999999999999997 , 0.9999999999999997
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 0.9999999999999997 , 0.9999999999999997
summation : 0.0threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 0.9999999999999997 , 0.9999999999999997
summation : 0.9999999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 0
Weights: 0.9999999999999997 , 0.9999999999999997
summation : 0.999x9999999999997threshold : 1
Actual Output : 0 Predicted Output : 0
---------------
Input : 1 , 1
Weights: 0.9999999999999997 , 0.9999999999999997
summation : 1.9999999999999993threshold : 1
Actual Output : 1 Predicted Output : 1
---------------
AND Gate Output for [1, 1] : 1
9. Develop a Python program for OR operation using perceptron.
CODE:
def activation(out,threshold):
if out>=threshold:
return 1
else:
return 0
def perceptron(or_input):
a=[0,0,1,1]
b=[0,1,0,1]
y=[0,1,1,1]
w=[0.0,0.3]
threshold=0.4
learning_rate=0.5
i=0
print("Perceptron Training : ")
print('######################')
print('------------')
while i<4:
summation = a[i]*w[0] + b[i]*w[1]
o= activation(summation,threshold)
print("Input : " + str(a[i]) + " , " + str(b[i]))
print("Weights: " +str(w[0]) + " , " + str(w[1]))
print("summation : "+str(summation) + "threshold : "+str(threshold))
print("Actual Output : "+str(y[i])+" Predicted Output : "+str(o))
if(o!=y[i]):
print("____________\nUpdating weights")
w[0]=w[0]+learning_rate*(y[i]-o)*a[i]
print("w[o]",w[o])
w[1]=w[1]+learning_rate*(y[i]-o)*b[i]
print("Updated Weights : " + str(w[0]) + " , "+ str(w[1]))
i=-1
print("\nWeights Updated Training Again : ")
print("###################################")
i=i+1
print("---------------")
summation = or_input[0]*w[0] + or_input[1]*w[1]
return activation(summation,threshold)
or_input = [1,1]
print("OR Gate Output for " +str(or_input) + " : "+str(perceptron(or_input)))
Output:
Perceptron Training :
######################
------------
Input : 0 , 0
Weights: 0.0 , 0.3
summation : 0.0threshold : 0.4
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 0.0 , 0.3
summation : 0.3threshold : 0.4
Actual Output : 1 Predicted Output : 0
____________
Updating weights
w[o] 0.0
Updated Weights : 0.0 , 0.8
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 0.0 , 0.8
summation : 0.0threshold : 0.4
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 0.0 , 0.8
summation : 0.8threshold : 0.4
Actual Output : 1 Predicted Output : 1
---------------
Input : 1 , 0
Weights: 0.0 , 0.8
summation : 0.0threshold : 0.4
Actual Output : 1 Predicted Output : 0
____________
Updating weights
w[o] 0.5
Updated Weights : 0.5 , 0.8
Weights Updated Training Again :
###################################
---------------
Input : 0 , 0
Weights: 0.5 , 0.8
summation : 0.0threshold : 0.4
Actual Output : 0 Predicted Output : 0
---------------
Input : 0 , 1
Weights: 0.5 , 0.8
summation : 0.8threshold : 0.4
Actual Output : 1 Predicted Output : 1
---------------
Input : 1 , 0
Weights: 0.5 , 0.8
summation : 0.5threshold : 0.4
Actual Output : 1 Predicted Output : 1
---------------
Input : 1 , 1
Weights: 0.5 , 0.8
summation : 1.3threshold : 0.4
Actual Output : 1 Predicted Output : 1
---------------
OR Gate Output for [1, 1] : 1
10. Build an artificial neural network by implementing the backpropagation algorithm and test the same using appropriate datasets.
CODE:
import numpy as np
X=np.array(([2,9],[1,5],[3,6]),dtype=float)
y=np.array(([92],[86],[89]),dtype=float)
X=X/np.amax(X,axis=0)
y=y/100
def sigmoid(x):
return 1/(1+np.exp(-x))
def derivatives_sigmoid(x):
return x*(1-x)
epoch=7000
lr=0.1
inputlayer_neurons=2
hiddenlayer_neurons=3
output_neurons=1
wh=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))
bh=np.random.uniform(size=(1,hiddenlayer_neurons))
wout=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))
bout=np.random.uniform(size=(1,output_neurons))
for i in range(epoch):
hinpl=np.dot(X,wh)
hinp=hinpl+bh
hlayer_act=sigmoid(hinp)
outinpl=np.dot(hlayer_act,wout)
outinp=outinpl+bout
output=sigmoid(outinp)
EO=y-output
outgrad=derivatives_sigmoid(output)
d_output=(EO*outgrad)
EH=d_output.dot(wout.T)
hiddengrad=derivatives_sigmoid(hlayer_act)
d_hiddenlayer=EH*hiddengrad
wout+=hlayer_act.T.dot(d_output)*lr
wh+=X.T.dot(d_hiddenlayer)*lr
print("Input: \n"+str(X))
print("Atual Output:\n"+str(y))
print("Predicted Output:\n",output)
Output:
Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]
Atual Output:
[[0.92]
[0.86]
[0.89]]
Predicted Output:
[[0.89304735]
[0.88208296]
[0.89538126]]0 views