Kids Online Brand Exposure Detection

Research Aim

This project investigates the feasibility of a semi-automated pipeline for detecting brand exposure in screen-recorded data. The framework integrates object detection, optical character recognition, and semantic reasoning to support large-scale analysis in scenarios where frame-level annotations are unavailable.

Rather than aiming for precise exposure measurement, the system is designed as a scalable screening tool — reducing manual review effort and producing outputs directly usable for downstream research analysis.

At a Glance

700K+ Video frames processed

4 Models in pipeline

2 yrs Real-world data covered

Semi Semi-automated pipeline

This project was developed as my Master's thesis at the University of Otago to support a large-scale study on how children are exposed to fast-food brand marketing through everyday screen-recorded media.

The core challenge: convert high-volume, unstructured video into structured, analysis-ready outputs — at a scale that made manual review completely impractical.

The Problem

Brand exposure in children's digital media hides in many forms — logos, product packaging, app interfaces, and background content. Manually reviewing hours of video is not scalable when the research goal requires consistent, reproducible detection across a large dataset spanning two years.

Pixel-based object detection alone also falls short. Logos appear at varying sizes, angles, and quality levels. Without a validation layer, ambiguous detections inflate false positives significantly.

How the Pipeline Works

Content-aware frame extraction Sample representative frames from raw video to reduce redundancy while preserving full temporal coverage.
YOLO-based logo detection Run YOLOv11 and YOLO-World models in parallel — both trained on brand logo data to detect visual matches across varied conditions.
OCR signal extraction Apply EasyOCR to capture brand names embedded in on-screen text, UI elements, and packaging copy.
CLIP semantic validation Apply zero-shot image-text similarity against brand-related prompts. Originally intended to reduce false positives, CLIP in practice also helped recover borderline true positives that YOLO alone would have missed.
Multimodal score fusion The YOLO ensemble score is combined with OCR text signals and CLIP semantic bonuses into an aggregated score. A final exposure decision is made only when this score exceeds the predefined threshold.
Exposure aggregation & structured output Frame-level detections are grouped into exposure events (detections of the same brand within a 2-second window are merged). Results are exported as CSV/XLSX files per recording, each containing brand name, start and end frame, exposure duration, and event frequency — ready for downstream research analysis.

YOLOv11 YOLO-World EasyOCR CLIP Python OpenCV Zero-shot learning Ensemble scoring Cloud processing

Evaluation: Threshold Tuning

A staged evaluation strategy was used to assess model behaviour across different confidence thresholds and ensemble configurations before running the pipeline on the full dataset.

Frame-level performance under different confidence thresholds

Precision, recall, and F1 score across confidence thresholds.

Selecting a threshold involved a deliberate trade-off: higher confidence reduces false positives but risks missing true exposures. The chart above guided the calibration decision.

Ensemble Tuning

Multiple combinations of YOLOv11 and YOLO-World ensemble weights and confidence thresholds were tested to find the configuration that best balanced detection accuracy against false positive rate. The final configuration adopted a 0.6:0.4 weight ratio at a confidence threshold of 0.50.

F1 score across ensemble weight configurations.

False positive rate across the same configurations.

Trade-off between false positives and F1 score

Trade-off chart used to select the final ensemble configuration.

Why a Hybrid Approach?

Each model covers a different failure mode:

YOLOv11 — high precision on known brand logos at pixel level.
YOLO-World — a second fine-tuned detection model that complements YOLOv11 and improves coverage across different visual conditions.
EasyOCR — catches brand names visible as on-screen text.
CLIP — zero-shot semantic matching against brand prompts; intended to filter false positives, but in practice also recovered borderline true positives where YOLO detection was marginal.

No single model reliably handles all the visual variation in real-world screen recordings. Combining them makes the pipeline more robust and reduces both missed detections and false positives.

How OCR & CLIP Combine with YOLO

Once the YOLO ensemble produces a detection score, OCR and CLIP are applied as supplementary signals — not as standalone detectors. Neither can trigger an exposure decision on its own. They work together in a two-tier confirmation mechanism:

Strong match — if OCR finds a brand keyword above 0.90 fuzzy confidence, or CLIP returns a semantic match above 0.90, and a corresponding YOLO detection exists, a direct confirmation bonus (+1.0) is applied, guaranteeing a positive decision.
Weak match — lower-confidence matches (OCR > 0.70, CLIP > 0.50) add a small bonus score to reinforce borderline YOLO detections without overriding the threshold.

The final exposure decision is made only when the aggregated score exceeds the predefined threshold. This keeps visual evidence (YOLO) as the dominant factor while allowing OCR and CLIP to strengthen ambiguous cases.

Figure 4.1 — Pipeline overview: Frame Input → YOLO detection → OCR & CLIP bonus signals → Aggregated score decision. (Source: thesis)

Sample Results

The following examples are drawn from real-world deployment on the Kids Online dataset. All recordings are anonymised.

True positive recovery cases — OCR and CLIP reinforcement examples

Figure 4.5 — (a) True positives recovered by OCR where visual-only detection failed. (b) True positives recovered through CLIP-based semantic reinforcement. Both observed across desktop and mobile platforms.

Residual failure modes — false positives requiring manual verification

Figure 4.6 — Residual failure modes observed during large-scale deployment, motivating the need for limited manual verification.

Outcome

The pipeline processed over 700,000 video frames and produced structured brand exposure outputs covering two years of real-world screen-recorded data.

It reduced the volume of footage requiring manual review and delivered results in a format directly usable for downstream research analysis and reporting — fulfilling its design goal as a scalable screening tool.

Limitations

Several limitations are acknowledged — primarily reflecting deliberate design trade-offs made to prioritise system-level feasibility and scalability.

Limited optimisation of auxiliary modules — OCR and CLIP were incorporated as supporting modules and were not extensively tuned. The primary focus was developing a stable YOLO-based ensemble framework.
Constrained effectiveness of CLIP under zero-shot settings — CLIP's contribution under full-frame, zero-shot configuration was limited. In some cases it overlapped with YOLO-World outputs or introduced additional false positives, suggesting that region-level reasoning would be more effective.
Simplified false positive handling — false positives were mainly controlled through ensemble filtering and confidence thresholds. More advanced suppression strategies were not implemented.
No temporal modelling — the pipeline operates at the frame level and does not model temporal continuity. Exposure events are treated as independent observations rather than coherent temporal segments.

These limitations do not undermine the feasibility of the proposed pipeline but define clear boundaries for its current scope.

Future Work

Several directions may further strengthen the pipeline:

Optimisation of OCR and CLIP components — image-level preprocessing (resolution adjustment, sharpness enhancement) and more systematic prompt engineering could better exploit CLIP's semantic reasoning capabilities.
Advanced false positive suppression — confidence reweighting or improved negative sample handling could further improve robustness under highly imbalanced conditions.
Temporal modelling — temporal smoothing or sequence-based aggregation could better capture exposure duration and persistence, improving interpretability in video-based detection tasks.
Alternative detection architectures — comparing the YOLO-based framework with grounding-based models (e.g., Grounding DINO) and evaluating model compression techniques for large-scale deployment feasibility.

Conclusion

This project contributes a reproducible and scalable framework for brand exposure analysis in real-world screen-recorded media. Rather than pursuing maximal detection accuracy under controlled conditions, it prioritises robustness and applicability — demonstrating that a pipeline can support large-scale exposure screening even without frame-level ground truth annotations.

By integrating detection-based and semantic approaches in a conservative, interpretable way, the work supports the broader goal of data-driven analysis of children's media exposure at scale — and provides a foundation for future research in this space.