Creative Artefact- A practical project to design and develop an ML prediction model with supporting

documentation

Assessmentlimits:

length, load,word count,

etc.

Assignment Description

This coursework aims to demonstrate the students' comprehensive understanding and knowledge of the Machine Learning module by evaluating their analytical abilities and strengths. It comprises two tasks designed to assess and challenge their analytical skills. These tasks have been carefully crafted to test the student's proficiency in applying the concepts learnedthroughout the moduleand to showcase their ability to tackle real-world scenarios and problems. Successful completion of these tasks willreflect the students' mastery of thesubject matter andtheir capacity for critical and creative thinking.

The Assessment Consists of TWO Tasks:

In Task 1, you will be presented with a set of questions that require critical evaluation of your subject knowledge and understanding of the concepts related to theMachine Learning (ML)process and techniques.

In Task 2, you will be provided a programming exercise with a dataset to analyse usingthe Machine Learning approach. Your objective is to use suitable MachineLearning algorithms to predict and evaluate a model's accuracy. The dataset will contain various features and a target variable thatyou need to predict. You are expected to develop a model, followed by training, testing, and evaluating its performance. Your goal is to identify and highlight the best predictive model for classification. This includes comparing different models and selecting the one that offers the highest accuracy and best performance metrics for the classification task.

The contentof this assignment must be supported by the inclusion of pertinent academic theories, concepts, models, and contemporary industrial insights. Provide a detailed and relevant description of your code. Ensure your work is accurately cited and referenced using the York St John Harvard Referencing Style.

Task 01: Theory Exercise – (20 Marks)

1. Discuss the K-means clustering algorithm in detail, including its working principles, advantages, disadvantages, and real-world applications. (10 Marks)

1. How would you evaluate the performance of the classification algorithm with appropriate metrics? (10 Marks)

Task 02: Programming Exercise -– (80 Marks)

a. Develop a classification modelwith Machine Learning techniques to detecthate speech fromthe Twitter dataset.

Dataset Description:

The "Hate Speech and Offensive Language" dataset is collected from Twitter. It is primarily designed to support research and development in detecting and analyzing hate speech and offensive language on social media, distinguishing them fromordinary slang andneutral content.

Dataset Link:

https://www.kaggle.com/datasets/mrmorj/hate-speech-and-offensive-language-dataset

Alternative Source:

The datasets are available to download fromthe Machine Learning (ML) module inthe Moodle platform.

Guidelines to Prepare Your Assignment:

1. Data Exploration and Pre-processing: (10 Marks)

• Load and explore the datasetto gain insights into the data'scharacteristics.
• Handle missing values, if any,and perform datacleansing as required.
• Perform datavisualization to understand variables' distribution and relationships withthem.

2. Feature Engineering Process: (15 marks)

• Extract meaningful information from the dataset usingFeature Extraction Technique
• Reduce the number of features usingPCA as the Dimensionality Reduction Technique
• Select themost prominent features to capture the complex relationships
• Handle the outliers to prevent themfrom significantly affecting the model

3. ModelSelection and Training: (10 Marks)

• Split thedataset into training and testing sets.
• Select appropriate machine learning algorithms (e.g., logistic regression, decision trees, random forests, support vector machines, etc.) for the predictive modeling task.
• Train the selected modelson the training data and evaluate their performance on the testing data.

4. Hyperparameter Tuning: (10 Marks)

• Fine-tune the hyperparameters of the chosen algorithm to optimize the model'sperformance and avoidmodel overfitting.

5. ModelEvaluation: (20 Marks)

• Compare the performance of different modelsusing appropriate evaluation metrics such as accuracy, precision, recall, F1-score, area under the receiver operating characteristic curve (AUC-ROC), Confusion matrix, and Logarithmic loss (Log Loss).
• Comparative analysis to identify the best-performing modelfor classification.

6. Model Deployment: (10 Marks)

• Deploy the model to generate predictions for new and previously unseendata.

7. Conclusion and Recommendations: (5 marks)

• Summarize the key findings of your analysis, highlighting the model'sperformance and any insights gained.

Propose potential improvements or additional steps that couldbe taken to enhance the system.

PLOs 7.1-7.7

1. Evaluate computer science concepts and principles and their application to the effective design, implementation, and usability of computer-based systems.
2. Apply the findings of advanced scholarship and/or contemporary research andpractice tothe solution to computer science problems
3. Critically evaluate computer science problems, including thoseat the forefront of the field.
4. Demonstrate operation within applicable professional, legal, social,and ethical frameworks.
5. Demonstrate originality and creativity in the solution of computer science problems.
6. Recommend, with detailed justification, the appropriate computer science principles and practices to apply to significant domain-specific activity.
7. Apply standards, qualityprocesses, and engineering principles to the solution of computer scienceproblems.

• Prepare     a     comprehensive    report     documenting    your     approach, methodologies, results, and insights gainedfrom the project.
• Include code snippets, visualizations, and explanations to support yourfindings. Your reportshould be clear,concise, and well-organized.
• Submit the complete report, including both Task 1 and Task 2, as a singlePDF file. The code must be submitted as a supporting document inthe .ipynb fileformat.

AdditionalGuidelines for Students:

Students must submit their own work. They must acknowledge the sources used in this assignment, failure to acknowledge would be plagiarism which is an academic offence and a penaltycan be imposed. Students need to write by reading other papers on their own with citations and leave references at the end of the assignment.

Students work would be submitted to the national plagiarism facility. This identifies the sources from the internet and other extensive databases. Once the student’s work has been submitted to detection services, work is stored in databases electronically and compared their work from other sources. It is necessary to keep a backup of their work.Students’ materials wouldbe stored in the database electronically for indefinite periods.

It is essential that you acknowledge the source of any research, information, ideas, opinions, theories, or other material which is not your own. Effective referencing,  quoting,  paraphrasing,  and  summarising show  evidence  of  the reading you have doneand ensure thatyou avoid accusations of plagiarism.

The University's fundamental stance on the use of Turnitin is geared toward supporting students' academic development. You can use this link to check your work for areas where you might be at risk of plagiarising.

Please submit your assignment on time. All assignments may be electronically submitted using Turnitin (via Moodle) by midnight on the due date. Please do not submityour assignment last minute. Pleasealso allow time for any problems or issues withsystems.

The work you present should be your own work, and not just copied from others.  You can quote from others, but you must say who the author is and use quotation marks or paraphrase.  If you do not do so, we will investigate your work for academic misconduct.  This is particularly likely if your Turnitin similarity score is above 25% and/or individual matches are above6%.

If you requiresupport with your study skills,please visit https://www.yorksj.ac.uk/students/study-skills/

Please refer to the York St John University Code of Practice for Assessmentand Academic RelatedMatters 2024-25.

We ask that you pay particular attention to the academic misconduct policy. Penalties will be applied where a student is found guilty of academic and/or ethicalmisconduct, including termination of programme (Policy Link).

You are required to keep to the word limit set for an assessment and to notethat you may be subject to penalty if you exceedthat limit. You are required to provide an accurate word count on the cover sheet for each piece of work you submit(Policy Link).

For late or non-submission of work by the published deadline or an approved extended deadline, a mark of 0NS will be recorded. Where a re-assessment opportunity exists, a student will normally be permitted only one attempt to be re-assessed for a cappedmark (Policy Link).

An extension to the published deadline may be granted to an individual student if theymeet the eligibility criteria of the (Policy Link).

Your work will be marked according to the assessment instructions provided within thisdocument and the selected Learning Outcomes’ (LOs) (see above).

Furthermore, this assessment is marked usingthe assessment marking criteria or a similar rubric that aligns with the University’s Generic Assessment Descriptors (see below).1 This is to ensure all assessment decisions are comparable regardless of the discipline or mode of assessment.

Please note that you must meet the required baseline standards (50 – 59%) which will include the LOs and minimum expectations of the assessment. Further still, you must ensure youmeet the requirements of each gradeboundary to progress to the next,i.e., you should demonstrate your learning through the standards of the Pass, Merit and Distinction to reach a Distinction (70 – 84%). These standards are designed to scaffold and build your learning to achieve yourfullest potential in each criterion being assessed.

Task 01- 20

Marks

An extraordinary conceptual understanding of K-meansclustering algorithm, advantages, and disadvantages with real-world applications. If anyexamples are

provided.

10

Marks

An appropriate description of all the metrics like accuracy, precision, recall, F1-score, area under the receiver operating characteristic curve (AUC-ROC), Confusion matrix, and Logarithmic

loss (Log Loss.)

10

Marks

Correct handling of missing values,

outliers,    and   data    normalization.

10

Marks

Jupyter       Notebook       or       scriptcontaining    the     code     for     data

preprocessing steps.Ensure that

80

Marks

Feature

Engineering

Present      code      segments      that

generate newfeatures based on

15

Marks

Correct implementation of selected

machine      learning      algorithms

10

Marks

10

Marks

5

Marks

100

Marks

Marking Criteria

Pass Grade Bands (100 – 50) (Learning Outcomes must be met)

Fail Grade Bands(49 – 0) (Learning Outcomes are not met)

Pass

(50 – 59)

Merit

(60 – 69)

Distinction

(70 – 84)

Distinction

(85 – 100)

Borderline Fail

(45 - 49)

(Credits may be compensated)

Fail

(30 - 44)

(Creditsmay not becompensated)

Fail

(0 - 29)

(Credits may not be compensate

d)

An

extraordin ary

Demonstrates a deep and

insightful understanding of the NB algorithm with

Shows a strong

understanding with relevant examples

Provides a good

understanding with some examples and

Demonstrates

adequate understanding with

Shows limited

understanding with insufficient examples

Demonstrates a

poor understanding

Fails to

demonstrat e

An innovative

approach to

Shows strong creativity with

effective problem- solving approaches.

Provides adequate creativity withbasic

problem-solving approaches.

Shows minimal creativity with

ineffective problem-solving.

Fails to demonstrat

e creativity or effective

Fails to handle data issueseffectively. Lacks proper EDA and documentati

on

Innovative and effective feature engineering with detailed explanation of encoding methods, mathematical

transformations, scaling, and feature selection.

Strong implementation with appropriate data splitting and effective use of

tools.

Poor implementation with ineffective data splitting or minimal tool

usage.

Fails to implement models correctly or use tools

effectively.

Fails to evaluate models effectively or provide meaningful

insights.

Fails to evaluate models effectively or provide

meaningful insights.

Model Deployment-10%

Fails to provide a coherent report or meaningful

analysis and