Load a Pandas dataframe with a selected dataset. Identify and count the missing values in a dataframe. Clean the data after removing noise as follows: a. Drop duplicate rows. b. Detect the outliers and remove the rows having outliers c. Identify the most correlated positively correlated attributes and negatively correlated attributes

 

Load a Pandas dataframe with a selected dataset. Identify and count the missing values in a
dataframe. Clean the data after removing noise as follows:
a. Drop duplicate rows.
b. Detect the outliers and remove the rows having outliers
c. Identify the most correlated positively correlated attributes and negatively correlated attributes



CODE



import pandas as pd


# Load the dataset

df = pd.read_csv('your_dataset.csv')


# Identify and count missing values

missing_values = df.isnull().sum()

print("Missing values:")

print(missing_values)


# Drop duplicate rows

df = df.drop_duplicates()


# Detect outliers and remove rows with outliers

def remove_outliers(df, column):

    Q1 = df[column].quantile(0.25)

    Q3 = df[column].quantile(0.75)

    IQR = Q3 - Q1

    lower_bound = Q1 - 1.5 * IQR

    upper_bound = Q3 + 1.5 * IQR

    return df[(df[column] >= lower_bound) & (df[column] <= upper_bound)]


columns_to_check = ['column1', 'column2']


for column in columns_to_check:

    df = remove_outliers(df, column)


# Identify the most correlated attributes

correlation_matrix = df.corr()


# Find the most positively correlated attributes

positive_corr = correlation_matrix.unstack().sort_values(ascending=False)

print("Most positively correlated attributes:")

print(positive_corr)


# Find the most negatively correlated attributes

negative_corr = correlation_matrix.unstack().sort_values(ascending=True)

print("Most negatively correlated attributes:")

print(negative_corr)


Comments

Post a Comment

Popular posts from this blog

The weights of 8 boys in kilograms: 45, 39, 53, 45, 43, 48, 50, 45. Find the median

  The weights of 8 boys in kilograms: 45, 39, 53, 45, 43, 48, 50, 45. Find the median To find the median, we need to arrange the weights in ascending order. Here are the weights of the 8 boys in kilograms: 39, 43, 45, 45, 45, 48, 50, 53 To find the median, we look for the middle value. Since there are 8 weights, the median is the average of the two middle values. In this case, the two middle values are 45 and 45. Therefore, the median weight is: (45 + 45) / 2 = 90 / 2 = 45 So, the median weight of the 8 boys is 45 kilograms. OR Here's how you can find the median of the given weights using Python: CODE:- weights = [45, 39, 53, 45, 43, 48, 50, 45] sorted_weights = sorted(weights) length = len(sorted_weights) if length % 2 == 0:     median = (sorted_weights[length // 2 - 1] + sorted_weights[length // 2]) / 2 else:     median = sorted_weights[length // 2] print("The median weight is:", median) When you run this code, it will output: The median weight is: 45.0 So, th...
Read more

Import iris data using sklearn library . Compute mean, mode, median, standard deviation, confidence interval and standard error for each feature ii. Compute correlation coefficients between each pair of features and plot heatmap iii. Find covariance between length of sepal and petal iv. Build contingency table for class feature

Import iris data using sklearn library or (Download IRIS data from: (https://archive.ics.uci.edu/ml/datasets/iris or import it from sklearn.datasets) i. Compute mean, mode, median, standard deviation, confidence interval and standard error for each feature ii. Compute correlation coefficients between each pair of features and plot heatmap iii. Find covariance between length of sepal and petal iv. Build contingency table for class feature CODE  import numpy as np import pandas as pd from sklearn.datasets import load_iris from scipy import stats import seaborn as sns import matplotlib.pyplot as plt # Load the Iris dataset iris = load_iris() data = iris.data feature_names = iris.feature_names target = iris.target target_names = iris.target_names # Convert data to a pandas DataFrame for easier analysis df = pd.DataFrame(data, columns=feature_names) df['class'] = target_names[target] # Compute mean, mode, median, standard deviation, confidence interval, and standard error for each f...
Read more