6. Baselines - ML Projects - Full Stack Deep Learning

Baselines act as a reality check for model performance by setting a lower bound on what a system can achieve. The tighter that lower bound, the more useful it becomes for deciding what to do next. Without a baseline, teams can misread progress: two models with identical training and validation curves can imply very different levels of success depending on where the baseline sits. If a blue curve representing “human performance” is near 20%, then a model that looks close to that curve suggests one kind of gap; if human performance is closer to 5%, the same model signals a much larger shortfall and different improvement priorities.

Baselines matter because they anchor interpretation. Error curves alone show relative movement, but they don’t tell whether the absolute level is good enough to justify further effort. That distinction drives downstream decisions—whether to invest in better architectures, more data, different labeling strategies, or entirely new approaches. In practice, teams can end up making the wrong call if they skip baselines and jump straight into training a sophisticated model (for example, a ResNet with aggressive data augmentation) under a tight deadline.

Where to find baselines depends on what resources are available. External sources include business or engineering requirements and published results, but comparisons must be fair: if the team’s dataset is harder than the one used in a paper, expecting the same performance is unrealistic. Internal baselines can be created quickly by scripting the problem using tools like OpenCV, building rule-based systems, or using simpler machine learning models such as bag-of-words or linear regression. The expectation is straightforward: a deep learning model should perform at least as well as these simpler references.

Human performance is another baseline, but it comes with a trade-off between data quality and ease of collection. Random labeling from people on Amazon Mechanical Turk is easy but often noisy. Quality improves by ensembling multiple random annotators—using agreement or majority vote rather than a single person’s answer. Domain experts raise the ceiling further, though they cost more time and effort. A similar ensemble approach can also be used among groups of experts.

Choosing where to land on that quality-versus-cost curve comes down to what the team will need next. The guidance is to use the highest-quality baseline that still allows iteration—especially because model improvement typically requires collecting more labeled data later. One practical tactic is to reserve the time of the most expensive labelers for the hardest examples, concentrating skilled effort where it yields the most information.

Finally, baselines support faster debugging and more disciplined decision-making. Teams can sometimes skip spending time on whether a project is worth doing or feasible when that decision is already made, but skipping baselines is riskier because it removes the reference point needed to interpret results. The discussion also notes a broader performance strategy: if feasibility is uncertain, it’s often better to first prove the task can be solved at all with a larger model, then meet speed constraints later through methods like pruning and knowledge distillation.