Faculty of Information Technology
FIT3152 Data analytics – 2023: Assignment 2
● The objective of this assignment is to gain familiarity with classification models using R.
● This is an individual assignment.
Suggested Length Due Date
Generative AI Use
● This assignment is worth 30% of your total marks for the unit.
● It has 35 marks in total.
• 8 – 10 A4 pages (for your report) + extra pages as appendix for your R script. • Font size 11 or 12pt, single spacing.
11.55pm Friday 12th May 2023
• In this assessment, you must not use generative artificial intelligence (AI) to generate any materials or content in relation to the assessment task.
Submission
• Submit a single PDF file and single video presentation file on Moodle.
• Use the naming convention: FirstnameSecondnameID.{pdf, mp4, mov etc.}
• Turnitin will be used for similarity checking of all written submissions.
Late Penalties
● 10% (3 mark) deduction per calendar day for up to one week.
● Submissions more than 7 calendar days after the due date will receive a
mark of zero (0) and no assessment feedback will be provided.
Instructions and data
The objective of this assignment is to gain familiarity with classification models using R.
We want to obtain a model that may be used to predict whether or not it will be more humid tomorrow than it is today for 10 locations in Australia.
You will be using a modified version of the Kaggle competition data: Predict rain tomorrow in Australia. https://www.kaggle.com/jsphyg/weather-dataset-rattle-package. The data contains meteorological observations as attributes, and the class attribute “more humid tomorrow”.
There are two options for compiling your written report:
(1) You can create your report using any word processor with your R code pasted in as machine- readable text as an appendix, and save as a pdf, or
(2) As an R Markdown document that contains the R code with the discussion/text interleaved. Render this as an HTML file and save as a pdf.
Your video report should be less than 100MB in size. You may need to reduce the resolution of your original recording to achieve this. Use a standard file format such as .mp4, or mov for submission.
Code Help
Creating your data set
Clear your workspace, set the number of significant digits to a sensible value, and use ‘WAUS’ as the default data frame name for the whole data set. Read your data into R and create your individual data using the following code:
rm(list = ls())
WAUS <- read.csv("HumidPredict2023D.csv")
L <- as.data.frame(c(1:49))
set.seed(XXXXXXXX) # Your Student ID is the random seed
L <- L[sample(nrow(L), 10, replace = FALSE),] # sample 10 locations
WAUS <- WAUS[(WAUS$Location %in% L),]
WAUS <- WAUS[sample(nrow(WAUS), 2000, replace = FALSE),] # sample 2000 rows
1. Explore the data: What is the proportion of days when it is more humid than the previous day compared to those where it is less humid? Obtain descriptions of the predictor (independent) variables – mean, standard deviations, etc. for real-valued attributes. Is there anything noteworthy in the data? Are there any attributes you need to consider omitting from your analysis?
2. Document any pre-processing required to make the data set suitable for the model fitting that follows.
3. Divide your data into a 70% training and 30% test set by adapting the following code (written for the iris data). Use your student ID as the random seed.
set.seed(XXXXXXXX) #Student ID as random seed train.row = sample(1:nrow(iris), 0.7*nrow(iris)) iris.train = iris[train.row,]
iris.test = iris[-train.row,]
4. Implement a classification model using each of the following techniques. For this question you may use each of the R functions at their default settings if suitable.
• Decision Tree
• Naïve Bayes
• Boosting
• Random Forest
5. Using the test data, classify each of the test cases as ‘more humid tomorrow’ or ‘less humid tomorrow’. Create a confusion matrix and report the accuracy of each model.
6. Using the test data, calculate the confidence of predicting ‘more humid tomorrow’ for each case and construct an ROC curve for each classifier. You should be able to plot all the curves on the same axis. Use a different colour for each classifier. Calculate the AUC for each classifier.
CS Help, Email: tutorcs@163.com
7. Create a table comparing the results in Questions 5 and 6 for all classifiers. Is there a single “best” classifier?
Investigative Tasks (18 Marks)
8. Examining each of the models, determine the most important variables in predicting whether it will be more humid tomorrow or not. Which variables could be omitted from the data with very little effect on performance? Give reasons. (2 Marks)
9. Starting with one of the classifiers you created in Question 4, create a classifier that is simple enough for a person to be able to classify whether it will be more humid tomorrow or not by hand. Describe your model, either with a diagram or written explanation. What factors were important in your decision? State why you chose the attributes you used. Using the test data created in Question 3, evaluate model performance using the measures you calculated for Questions 5 and 6. How does it compare to those in Question 4? (4 Marks)
10. Create the best tree-based classifier you can. You may do this by adjusting the parameters, and/or cross-validation of the basic models in Question 4. Show that your model is better than the others using the measures you calculated for Questions 5 and 6. Describe how you created your improved model, and why you chose that model. What factors were important in your decision? State why you chose the attributes you used. (4 Marks)
11. Using the insights from your analysis so far, implement an Artificial Neural Network classifier and report its performance. Comment on attributes used and your data pre- processing required. How does this classifier compare with the others? Can you give any reasons? (4 Marks)
12. Fit a new classifier to the data, test and report its performance in the same way as for previous models. You can choose a new type of classifier not covered in the course, or a new version of any of the classifiers we have studied. Either way, you will be implementing a new R package. As a starting point, you might refer to James et al. (2021), or look online. When writing up, state the new classifier and package used. Include a web link to the package details. Give a brief description of the model type and how it works. Comment on the performance of your new model. (4 Marks)
Report and Video Presentation (7 Marks)
Write a brief report (suggested length 8 – 10 pages) summarizing your results. Use commenting in your R script, where appropriate, to help a reader understand your code. Alternatively combine working, comments and reporting in R Markdown. (3 Marks)
Record a short presentation using your smart phone, Zoom, or similar method. Your presentation should be approximately 5 minutes in length and summarise your main findings, as well as describing how you conducted your research and any assumptions made. Pay particular emphasis to your results for the investigative tasks. (Submission Hurdle and 4 Marks)
It is expected that you will use R for your data analysis and graphics and tables. You are free to use any R packages you need but please document these in your report and include in your R code.
Description of the data
Attribute 1, Year
Attribute 2, Location
Attribute 3, MinTemp
Attribute 4, MaxTemp
Attribute 5, Rainfall
Attribute 6, Evaporation
Attribute 7, Sunshine
Attribute 8, WindGustDir
Attribute 9, WindGustSpeed
Attribute 10, WindDir9am
Attribute 11, WindDir3pm
Attribute 12, WindSpeed9am
Attribute 13, WindSpeed3pm
Attribute 14, Pressure9am
Attribute 15, Pressure3pm
Attribute 17, Cloud3pm
Attribute 18, Temp9am
Attribute 19, Temp3pm
Attribute 20, RainToday
Attribute 21, RISK_MM
Attribute 22, MHT
Year of the observation.
The location of the observation. The daily minimum temperature in degrees Celsius.
The daily maximum temperature in degrees Celsius.
Rainfall recorded for the day in mm. The evaporation (mm) in the 24 hours to 9am.
Hours of bright sunshine over the day.
Direction of strongest wind gust over the day.
Speed (km/h) of the strongest wind gust over the day.
Direction of the wind at 9am. Direction of the wind at 3pm.
Speed (km/hr) averaged over 10 minutes prior to 9am.
Speed (km/hr) averaged over 10 minutes prior to 3pm.
Atmospheric pressure (hpa) reduced to mean sea level at 9am. Atmospheric pressure (hpa) reduced to mean sea level at 3pm.
Fraction of sky obscured by cloud at 3pm.
Temperature (degrees C) at 9am. Temperature (degrees C) at 3pm.
1 if precipitation (mm) in the 24 hours to 9am exceeds 1mm, otherwise 0.
The amount of rain. A kind of measure of the "risk".
The target variable. Will it be more humid tomorrow (MHT)than today?
Attribute 16, Cloud9am
Fraction of sky obscured by cloud at 9am. This is measured in "oktas", which are a unit of eigths. It records how many eigths of the sky are obscured by cloud. A 0 measure indicates completely clear sky whilst an 8 indicates that it is completely overcast.
References
An Introduction to Statistical Learning with applications in R, 2nd Ed, 2021. (Springer Texts in Statistics), James, Witten, Hastie and Tibshirani. (Available on-line from Monash Library.)
程序代写 CS代考 加微信: cstutorcs