Load a Pandas dataframe with a selected dataset. Identify and count the missing values in a dataframe. Clean the data after removing noise as follows: a. Drop duplicate rows. b. Detect the outliers and remove the rows having outliers c. Identify the most correlated positively correlated attributes and negatively correlated attributes

 

Load a Pandas dataframe with a selected dataset. Identify and count the missing values in a
dataframe. Clean the data after removing noise as follows:
a. Drop duplicate rows.
b. Detect the outliers and remove the rows having outliers
c. Identify the most correlated positively correlated attributes and negatively correlated attributes



CODE



import pandas as pd


# Load the dataset

df = pd.read_csv('your_dataset.csv')


# Identify and count missing values

missing_values = df.isnull().sum()

print("Missing values:")

print(missing_values)


# Drop duplicate rows

df = df.drop_duplicates()


# Detect outliers and remove rows with outliers

def remove_outliers(df, column):

    Q1 = df[column].quantile(0.25)

    Q3 = df[column].quantile(0.75)

    IQR = Q3 - Q1

    lower_bound = Q1 - 1.5 * IQR

    upper_bound = Q3 + 1.5 * IQR

    return df[(df[column] >= lower_bound) & (df[column] <= upper_bound)]


columns_to_check = ['column1', 'column2']


for column in columns_to_check:

    df = remove_outliers(df, column)


# Identify the most correlated attributes

correlation_matrix = df.corr()


# Find the most positively correlated attributes

positive_corr = correlation_matrix.unstack().sort_values(ascending=False)

print("Most positively correlated attributes:")

print(positive_corr)


# Find the most negatively correlated attributes

negative_corr = correlation_matrix.unstack().sort_values(ascending=True)

print("Most negatively correlated attributes:")

print(negative_corr)


Comments

Post a Comment

Popular posts from this blog

Discuss in detail about Data mining Applications.

Data mining has a wide range of applications across various industries, contributing to informed decision-making, pattern discovery, and knowledge extraction from large datasets. Here are some detailed discussions on the applications of data mining in different domains: Business and Marketing: Market Basket Analysis: Data mining is extensively used in retail for market basket analysis. It helps identify associations between products that are frequently purchased together, enabling businesses to optimize product placements, plan promotions, and enhance cross-selling strategies. Customer Segmentation: Businesses use data mining to segment their customer base based on various characteristics such as purchasing behavior, demographics, or preferences. This information helps in targeted marketing and personalized customer experiences. Healthcare: Disease Prediction and Diagnosis: Data mining is employed to analyze medical records, diagnostic data, and patient history to predict the likelihoo...
Read more

Implement stack data structure and its operations using arrays.

Implement stack data structure and its operations using arrays.  #include <iostream> using namespace std; const int MAX_SIZE = 100; class Stack { private:     int top;            // Index of the top element in the stack     int arr[MAX_SIZE];  // Array to store the stack elements public:     Stack() {         top = -1;       // Initialize top to -1 to indicate an empty stack     }     bool isEmpty() {         return (top == -1);     }     bool isFull() {         return (top == MAX_SIZE - 1);     }     void push(int value) {         if (isFull()) {             cout << "Stack Overflow: Cannot push element " << value << ". Stack is full." << endl;             return;   ...
Read more