--- title: "BIF20806_week1day2_notebook" editor: source date: ADD author: ADD registration_number: ADD format: html execute: eval: false --- # From messy to clean ## Set up your R environment ```{r setup} options(repos = c(CRAN = "https://cloud.r-project.org")) ### Some of you will have problems with setting the workdirectory (setwd()) ### Here you need to set your work directory as well #knitr::opts_knit$set(root.dir = "D:/") ``` ### Set a work directory Here you can add notes to help yourself in the future. ```{r} ### Hint: you can copy/paste from previous scripts ###Note that you need to set it as well in the {r setup} chunk at the start of this document. ``` ### Packages Here you can add notes to help yourself in the future. ```{r} ``` ## Loading data into R ### Loading data from an excel file (.xlsx) (excercise) Here you can add notes to help yourself in the future. ```{r} ``` ### Loading data from a CSV File (Comma-separated) (excercise) Here you can add notes to help yourself in the future. ```{r} ``` ### Loading data from a CSV file (semicolon-separated) (excercise) Here you can add notes to help yourself in the future. ```{r} ``` ### Loading a Tab-delimited File (excercise) Here you can add notes to help yourself in the future. ```{r} ``` ## Inspecting data part 1 ### Using functions Here you can add notes to help yourself in the future. ```{r} # Check the first 6 rows head(data_csv) # Check the last 6 rows tail(data_csv) # get a summary of the data summary(data_csv) # get the length of a vector length(data_csv) # get the number of columns ncol(data_csv) # get the number of rows nrow(data_csv) # get the dimensions of an object dim(data_csv) ``` ### Which file is which puzzle piece? (excercise) Inspect the four files you have using `dim()`, `nrow()`, and `ncol()`. Which piece is which? Answer the following questions (note this is *random* for each personal dataset!). You can deduce which piece is which via their dimensions. 1. Which 2 files are piece 1 and 2? 2. which file is piece 3? 3. Which file is piece 4? ## Cleaning and preparing the data ### Combine the data (excercise) Here you can add notes to help yourself in the future. ```{r} ### bind objects together over columns pheno_data <- cbind(YOURPIECE1,YOURPIECE2) ### binds objects together over rows pheno_data <- rbind(pheno_data,YOURPIECE3) ### merges objects, seeks for the same identitier, or you can provide it using by.x and by.y pheno_data <- merge(pheno_data,YOURPIECE4) ``` ### Inspecting data using histograms Here you can add notes to help yourself in the future. ```{r} ggplot2::ggplot(pheno_data,aes(x=Gpal_vir)) + geom_histogram() ggplot2::ggplot(pheno_data,aes(x=Gpal_avir)) + geom_histogram() ``` ## Analyzing ### Normal distribution Here you can add notes to help yourself in the future. ```{r} ### Using a plot ### Using a statistical test ``` ### Does the data follow a normal distribution? (exercise) You now conducted two types of analyses that tell you whether the data is normally distributed. (1) what do you see in the qqplots, and what does it mean? (2) what are the p-values from the Shapiro-Wilk test? (3) what does a p-value mean? ### Wilcoxon test Here you can add notes to help yourself in the future. ```{r} ### prepare the data head(pheno_data) data.test <- tidyr::gather(pheno_data,key="pallida",value="juveniles",-ID,-Experiment) head(pheno_data) ### Conduct the statistical test wilcox.test(juveniles~pallida,data=data.test) ###you should have a very low p-value ``` ### How to interpret a p-value? (exercise) You now have tested a null-hypothesis using a non-parametric test and found that the chance his data agrees with this null-hypothesis is very small. What is the *biological interpretation* of this p-value? ## Presenting ### Boxplot Here you can add notes to help yourself in the future. ```{r} ### Simple boxplot ggplot(data.test,aes(x=pallida,y=juveniles)) + geom_boxplot() + geom_jitter(height=0,width=0.25) ``` ### Boxplot with facet_grid() (exercise) Here you can add notes to help yourself in the future. ```{r} ``` #### Fancier boxplot Here you can add notes to help yourself in the future. ```{r} ###I'm getting the statistics here tmp <- tapply(data.test[,c("pallida","juveniles")],data.test$Experiment,function(x){ broom::tidy(wilcox.test(x$juveniles~x$pallida))}) ### I need to include the facets statplot <- do.call(rbind,tmp) %>% mutate(Experiment=1:3,pallida=1.5,juveniles=max(data.test$juveniles), label=paste("p =",signif(p.value,2))) ###save the plot into an object p1 <- ggplot(data.test,aes(x=pallida,y=juveniles)) + geom_boxplot() + geom_jitter(height=0,width=0.25) + facet_grid(~Experiment) + geom_text(aes(label=label),data=statplot) ###plot the plot p1 ``` ### Save the plot (exercise) Use a graphical device to save the plot with dimensions of 7 (width) by 4 (height). ```{r} ``` Confirm that you now have a file on your HDD, within the folder you set as your working directory. ## Inspecting data part 2 ### Correlation analysis Here you can add notes to help yourself in the future. ```{r} data.test <- pheno_data ### Pearson correlation cor(data.test$Gpal_vir,data.test$Gpal_avir) ### Spearman correlation cor(data.test$Gpal_vir,data.test$Gpal_avir,method = "spearman") ### Test of correlation ### it is possible you get a warning here; this is not an error. cor.test(data.test$Gpal_vir,data.test$Gpal_avir,method = "spearman") ``` ### Correlation analysis interpretation (exercise) What do you conclude from the correlation analysis, are the two values correlated? What is the *biological* interpretation of this result? ### Clustering analysis Here you can add notes to help yourself in the future. ```{r} ### To make the distance matrix, we first need to summarise the data ### otherwise the matrix becomes too large data.test <- dplyr::group_by(pheno_data,Experiment) %>% dplyr::summarise(Gpal_vir=mean(Gpal_vir),Gpal_avir=mean(Gpal_avir)) ###check the shape of the data before gather head(data.test) data.test <- tidyr::gather(data.test,key=population,value=reproduction,-Experiment) ###and after gather head(data.test) ### Here we calculate the distance matrix dist_mat <- dist(data.test$reproduction,diag=TRUE,upper=TRUE) ### to give understandable names to the distance matrix we re-write it as matrix dist_mat <- as.matrix(dist_mat) ### these two lines replace the column and row names in the matrix colnames(dist_mat) <- paste(data.test$population,data.test$Experiment) rownames(dist_mat) <- paste(data.test$population,data.test$Experiment) ### to conduct clustering the matrix is re-written als distance matrix dist_mat <- as.dist(dist_mat) ### here we use the base plot-function to look at the clustering plot(hclust(dist_mat)) ``` ### Clustering analysis interpretation (exercise) What do you conclude from the clustering analysis, are there differences between the two *G. pallida* populations? What is the *biological* interpretation of this result? ### heatmap Here you can add notes to help yourself in the future. ```{r} ### To make the distance matrix, we first need to summarise the data ### otherwise the matrix becomes too large data.test <- dplyr::group_by(pheno_data,Experiment) %>% dplyr::summarise(Gpal_vir=mean(Gpal_vir),Gpal_avir=mean(Gpal_avir)) %>% tidyr::gather(key=population,value=reproduction,-Experiment) ### Here we calculate the distance matrix dist_mat <- dist(data.test$reproduction,diag=TRUE,upper=TRUE) ### to give understandable names to the distance matrix we re-write it as matrix dist_mat <- as.matrix(dist_mat) ### these two lines replace the column and row names in the matrix colnames(dist_mat) <- paste(data.test$population,data.test$Experiment) rownames(dist_mat) <- paste(data.test$population,data.test$Experiment) heatmap(dist_mat) ``` ### Plot of correlation (exercise) Add the correct column name for the y-axis and the right geom to plot all data points. ```{r} ```