Outlier Report

Project roadmap

1. 01

   Define inputs and outputs

   ~1h

   Write README.md: the input is a CSV file plus the names of the numeric columns to check; the output is outliers.csv with the row number, column, value, rule used and the limit it crossed, and one box plot per column. Choose a sample dataset with a few obvious outliers, or add some yourself. Install the tidyverse and shiny packages. Done when: README.md describes the input, the output and the three rules.

2. 02

   Load and check the data

   ~1.5h

   Load the file with readr::read_csv and check that the requested columns exist and are numeric, stopping with a clear message otherwise. Count missing values per column and leave them out of the calculations (without treating them as outliers). Done when: loading a file with a wrong column name or a text column gives a helpful error.

3. 03

   Implement the rules

   ~2h

   Write a function find_outliers(data, columns, rule, ...) supporting three rules: iqr (below Q1 − 1.5×IQR or above Q3 + 1.5×IQR, with the multiplier adjustable), zscore (more than 3 standard deviations from the mean, adjustable) and range (fixed minimum and maximum per column). Return a tibble with the row number, column, value, rule and limit. Use dplyr and tidyr (pivot_longer helps handle many columns at once). Done when: each rule flags the outliers you planted in the sample data.

4. 04

   Write the report and charts

   ~1.5h

   Save the flagged rows to outliers.csv, and print a summary per column: the number of values, missing values and outliers. Draw one box plot per column with ggplot2 (facet_wrap), with outliers highlighted in color, and save it as a PNG. Done when: running the script creates the CSV and the chart, and they match each other.

5. 05

   Build a small Shiny app

   ~2h

   Create app.R with a file upload, a choice of numeric columns, a rule selector and its setting (the multiplier, or the minimum and maximum), a table of flagged rows and the box plot, reusing your find_outliers function. Add a button to download the flagged rows as CSV. Done when: uploading the sample file and changing the rule updates the table and chart.

6. 06

   Test edge cases

   ~1.5h

   Write tests with testthat: a column where every value is the same (standard deviation of 0 must not cause errors), a column with only missing values, very small datasets, negative numbers, and each rule on a tiny hand-made dataset with a known answer. Done when: all tests pass.

Resources

• DocsR for Data Science (free online book)
• ArticleOutlier (Wikipedia)
• Docsggplot2 geom_boxplot reference
• DocsMastering Shiny (free online book)
• Docstestthat documentation