Statistics for Research - L12 - How to Identify and Deal with Outliers in R?

Outliers—unusually high or low responses—can distort research results by pulling summary statistics like the mean away from what most participants actually report. A classic example is a coffee-consumption survey where nearly everyone answers between 0 and 4 cups per day, but one participant reports 17 cups. That single extreme value can inflate the sample mean and make it look like the “average” coffee drinker consumes far more than the typical respondent, even if the outlier is due to a caffeine issue or a data-entry mistake.

The session lays out practical ways to spot outliers before deciding what to do with them. One approach is to inspect descriptive statistics—especially minimum and maximum values—to see whether any observations fall far outside the expected range. Visual checks also matter: histograms, box plots, and bar graphs can reveal points that sit apart from the rest. For a more standardized method, responses can be converted into Z scores, which rescale values so they have a mean of 0 and a standard deviation of 1. In this framework, values with Z scores greater than about 3.3 (or less than -3.3) are treated as outliers, making it easier to compare across cases and variables.

Once outliers are identified, the key decision is whether they are meaningful or spurious. If an extreme response makes no sense in context—such as a value that appears to be a mistake—or is unjustifiably extreme, removing it can be appropriate. If the response is plausible and only slightly beyond the threshold, keeping it may be safer. The session also lists several handling options: ignore outliers when they don’t materially affect results; trim them by removing erroneous observations (common when they stem from measurement or entry errors); reduce their influence using transformations like taking the log of the data; or replace them using percentile-based values (for example, substituting with the 5th or 95th percentile).

The practical R workflow is then demonstrated. Data is loaded from a CSV file using `read.csv` with headers enabled. After viewing and checking the data, `summary()` is used to find minimum and maximum values for a selected variable. If extremes appear, `which()` helps locate the exact observations that violate expected bounds (e.g., values greater than 7). To apply the Z-score rule, the session uses `scale()` to compute standardized scores and again uses `which()` to identify observations whose Z scores exceed the threshold.

A worked example follows: after finding two problematic observations for `Vision one`, the script determines their column index and observation numbers, then corrects the entries by referencing the original questionnaire (changing values like 16 and 14 to 6 and 4). After writing the corrected data back to the file, rerunning the checks shows that the outliers disappear. The session closes with the takeaway that outlier handling should be deliberate—because removing or ignoring points without justification can lead to incorrect conclusions—and that R can support both detection and correction.