hw-6

Homework 6 in DSCI445: Statistical Machine Learning @ CSU

Assignment

Note: tidymodels does not include a way to perform subset selection. Here is a useful list of functions for the methods in this chapter:

• regsubsets in the leaps package performs subset selection methods.
• linear_reg(mixture = 0, penalty = lambda) |> set_mode("regression") |> set_engine("glmnet") defines ridge regression with penalty \(\lambda\).
• linear_reg(mixture = 1, penalty = lambda) |> set_mode("regression") |> set_engine("glmnet") defines the lasso with penalty \(\lambda\).

Be sure to set.seed(445).

1. In this exercise, we will generate simulated data, and then use this data to perform best subset selection.

   1. Use rnorm to generate a predictor \(X\) of length \(n = 100\) and a noise vector \(\epsilon\) also f length \(n = 100\).

   2. Generate a response vector \(Y\) of length \(n = 100\) according to the model \[ Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 X^3 + \epsilon \] where \(\beta_0 = 1, \beta_1 = -0.5, \beta_2 = 2, \beta_3 = -1\).

   3. Use the regsubsets function in the leap package to perform best subset selection in order to choose the best model containing the predictors \(X, X^2, \dots, X^{10}\). What is the best model obtained according to \(C_p\), BIC, and Adjusted \(R^2\)? Show some plots to provide evidence for your answer and report the coefficients of the best model obtained.

      [Hint 1: The poly function may be useful for creating the model formula.]

      [Hint 2: You will need to make a data frame with your X and Y variables.]

   4. Repeat c. using forward stepwise selection and also using backwards stepwise selection. How does your answer compare to the results in c.?

   5. Now fit a lasso model to the simulated data using \(X, X^2, \dots, X^{10}\) as predictors. Use \(10\)-fold CV to choose the optimal value of \(\lambda\). Create plots of the CV error as a function of . Report the resulting coefficient estimates and discuss the results obtained.

2. In this exercise we will predict the number of applications received using the other variables in the College data set (in the ISLR package).

   1. Split the data into training (60%) and “test” (40%) set randomly.

   2. Fit a linear model using least squares on the training set and report the test set error obtained.

   3. Fit a ridge regression model on the training set with \(\lambda\) chosen using 10-fold CV (on the training set only). Report the test error obtained.

   4. Fit the lasso on the training set with \(\lambda\) chosen using 10-fold CV (on the training set only). Report the test error obtained, along with the number of non-zero coefficient estimates.

   5. Comment on the results obtained. How acurately can we predict the number of college applications received? Is there much difference among the test errors resulting from these five approaches?

Turn in in a pdf of your homework to canvas using the provided Rmd file as a template. Your Rmd file on the server will also be used in grading, so be sure they are identical.

Be sure to share your server project with the instructor and grader. You only need to do this once per semester.

1. Open your homeworks project on liberator.stat.colostate.edu

2. Click the drop down on the project (top right side) > Share Project…

   

3. Click the drop down and add “dsci445instructors” to your project.

   

This is how you receive points for reproducibility on your homework! on the server will also be used in grading, so be sure they are identical.