Blog & Notes

Good References

Brainstorm

In the context of machine learning knowledge distillation, training both the teacher model and the student model in parallel is generally not the standard or most effective approach, though it can make sense in specific scenarios depending on the goals and constraints.

1. Standard Knowledge Distillation Process

Knowledge distillation typically involves a two-stage process:

1. Teacher Model Training: A large, complex model (the teacher) is trained first on the target task using the available dataset. The teacher model is usually overparameterized and achieves high performance, but may be computationally expensive.
2. Student Model Training: A smaller, more efficient model (the student) is then trained to mimic the teacher's behavior. This is done by using the teacher's outputs (e.g., softened logits or feature representations) as soft targets, in addition to or instead of the ground-truth labels. The student is trained to minimize a loss function that includes a distillation loss (e.g., Kullback-Leibler divergence between the teacher's and student's outputs) and, often, a task-specific loss (e.g., cross-entropy with ground-truth labels).

The teacher is typically pre-trained and fixed during the student's training because the goal is to transfer the teacher's learned knowledge to the student. The teacher's outputs provide a richer supervision signal (e.g., capturing inter-class relationships via soft probabilities) than hard labels alone.

2. Why Parallel Training Is Uncommon

Training the teacher and student models in parallel (i.e., simultaneously updating both models during training) is not standard for several reasons:

1. Teacher Stability: The teacher model is expected to provide reliable and high-quality outputs to guide the student. If the teacher is still training and its parameters are changing, its outputs may be noisy or inconsistent, which could destabilize the student's learning process.
2. Computational Overhead: Training both models simultaneously increases computational requirements, as both models need to perform forward and backward passes. Knowledge distillation aims to produce a lightweight student model for efficiency, so adding the overhead of training a large teacher in parallel contradicts this goal.
3. Objective Misalignment: The teacher is typically optimized for task performance (e.g., minimizing cross-entropy loss), while the student is optimized to mimic the teacher's outputs (distillation loss) and, optionally, the ground-truth labels. Jointly optimizing both models could lead to conflicting gradients or objectives, making convergence harder.
4. Sequential Dependency: The student relies on the teacher's knowledge, which is most effective when the teacher has already converged on a good solution. Training them in parallel undermines this dependency, as the teacher may not yet have reliable knowledge to transfer.

Scenarios Where Parallel Training Might Make Sense

There are specific cases where training the teacher and student in parallel could be considered, though these are less common and typically involve modifications to the standard distillation framework:

1. Online Knowledge Distillation (OKD): In online distillation (e.g., as proposed in methods like Deep Mutual Learning), multiple models (which can be seen as peers rather than a strict teacher-student hierarchy) are trained simultaneously, and they learn from each other's outputs. Instead of a pre-trained teacher, all models act as both teachers and students, sharing knowledge via their predictions. Use Case: Scenarios with limited access to a pre-trained teacher or when training multiple models of similar capacity to improve generalization.
2. Dynamic Teacher Updates: In some setups, the teacher model may be fine-tuned or updated during training to adapt to the student's needs or to incorporate new data. This requires careful design to ensure the teacher remains a stable source of knowledge, such as using a slower learning rate for the teacher or alternating updates. Use Case: Continual learning, online learning, or scenarios with streaming data where pre-training a teacher is impractical.
3. Co-Training for Efficiency: If computational resources allow, parallel training could be explored to reduce the total training time compared to sequential training. For example, the teacher and student could share some computational graphs (e.g., in a shared feature extraction backbone) to save resources. Use Case: Resource-rich environments where training time is a bottleneck.
4. Reinforcement Learning or Adversarial Settings: In some advanced setups, such as those inspired by GANs or reinforcement learning, the teacher and student might be trained competitively or cooperatively. Use Case: Research settings exploring novel training dynamics or adversarial knowledge transfer.

Challenges of Parallel Training

• Stabilizing Teacher Outputs: The teacher's outputs may be noisy early in training, so techniques like output smoothing, teacher ensembling, or delayed distillation may be needed.
• Balancing Objectives: The teacher and student may have different learning rates, loss weights, or objectives, requiring careful tuning.
• Increased Complexity: Parallel training requires more memory and compute, which may not be feasible for large teacher models.
• Evaluation Metrics: You'd need to define how to evaluate the teacher's readiness to provide useful knowledge.

Alternative Approaches

• Self-Distillation: The student model distills knowledge from itself (e.g., from earlier layers or epochs) rather than a separate teacher.
• Pre-Trained Teacher Reuse: Use an existing pre-trained model (e.g., a foundation model like BERT or a publicly available checkpoint) as the teacher.
• Online Distillation Variants: Explore methods like Deep Mutual Learning or Collaborative Learning, where multiple models learn together without a fixed teacher-student hierarchy.

Practical Recommendation

In most cases, sequential training (pre-training the teacher, then training the student) is the most effective and straightforward approach for knowledge distillation. It ensures the teacher provides stable, high-quality outputs and aligns to produce an efficient student model. Parallel training could be explored in research settings or specific use cases (e.g., online distillation or continual learning), but it requires careful design to manage the challenges outlined above.

3. Why Pretraining the Teacher and Training the Student on the Same Dataset Makes Sense

Consistency in Data Distribution

Using the same dataset for both the teacher and student ensures that the knowledge being transferred is relevant to the task and data distribution that the student will encounter. The teacher learns the patterns, relationships, and structure of the dataset, and the student benefits from mimicking these learned representations. If the teacher were trained on a different dataset, its knowledge might not generalize well to the student's target dataset, leading to suboptimal performance.

Teacher as a High-Quality Reference

Pretraining the teacher model on the same dataset allows it to converge to a high-performing solution, capturing rich information about the task (e.g., class probabilities, feature representations, or decision boundaries). The student can then leverage this high-quality knowledge via the teacher's soft outputs (e.g., softened logits or intermediate features), which provide more information than hard labels alone. For example, in classification tasks, the teacher's softmax outputs (with a temperature parameter) reveal inter-class relationships, helping the student learn nuanced decision boundaries.

Standard Practice in Knowledge Distillation

The canonical knowledge distillation framework, as introduced by Hinton et al. (2015), assumes the teacher is pretrained on the target dataset, and the student is trained to match the teacher's outputs on the same dataset. This setup is widely used in applications like model compression for image classification, natural language processing, and speech recognition. The student typically optimizes a combination of the distillation loss (e.g., KL divergence between the teacher's and student's logits) and the task-specific loss (e.g., cross-entropy with ground-truth labels) on the same dataset.

Efficient Use of Data

In many real-world scenarios, the dataset is fixed and limited. Using the same dataset for both models avoids the need for additional data collection or the risk of domain mismatch. The teacher extracts as much knowledge as possible from the dataset, and the student distills this knowledge into a more compact model.

Benefits of This Approach

• Improved Student Performance: The student benefits from the teacher's learned knowledge, often achieving better performance than if trained solely on ground-truth labels.
• Generalization: The teacher's soft targets act as a form of regularization, helping the student generalize better by learning smoother decision boundaries.
• Flexibility: This approach works across various tasks (e.g., classification, regression, sequence modeling) and model architectures.
• Data Efficiency: Even with limited labeled data, the teacher's outputs provide additional supervision, making it easier for the student to learn effectively.

Considerations and Best Practices

1. Teacher Quality: The teacher must be well-trained and perform well on the dataset. A poorly trained teacher will provide noisy or unreliable outputs, leading to suboptimal student performance.
2. Distillation Loss Design: The student's loss function typically combines the distillation loss (matching the teacher's outputs) and the task-specific loss (matching ground-truth labels). The balance (controlled by a weighting hyperparameter, e.g., α in L = α * L_distillation + (1-α) * L_task) is critical. The temperature parameter in the softmax function also needs tuning — a higher temperature (e.g., T = 2 or T = 4) makes the teacher's outputs smoother.
3. Overfitting Risk: If the dataset is small, the teacher might overfit. Use regularization techniques during teacher training (e.g., dropout, weight decay), data augmentation, and monitor the teacher's generalization performance.
4. Dataset Size and Diversity: For very small datasets, the teacher's knowledge may be limited. Consider using a pre-trained teacher or augmenting the dataset.
5. Model Architecture Compatibility: Ensure the teacher and student models are compatible in terms of their output spaces. If the student has a different architecture, additional techniques like feature-based distillation may be needed.
6. Computational Efficiency: Since the teacher is pretrained, its parameters are fixed during student training. The student only needs the teacher's outputs, which can be precomputed and cached for the entire dataset to save time.

Potential Alternatives or Variations

• Transfer Learning or Domain Adaptation: If the teacher is pretrained on a larger, related dataset (e.g., ImageNet), it can be fine-tuned on the target dataset before distillation.
• Unlabeled or Augmented Data: In some cases, the student can be trained on a mix of labeled data and unlabeled data, where the teacher provides pseudo-labels for the unlabeled data.
• Online Distillation: If you want to avoid pretraining the teacher entirely, you could explore online knowledge distillation (e.g., Deep Mutual Learning).

Practical Example

Suppose you're working on an image classification task with the CIFAR-10 dataset:

• Teacher Training: Train a large model (e.g., ResNet-50) on CIFAR-10 until it achieves high accuracy (e.g., 95% on the test set).
• Student Training: Train a smaller model (e.g., MobileNet) on CIFAR-10, using a loss function that combines KL divergence between the teacher's softened logits (temperature T=4) and the student's logits, plus cross-entropy loss with ground-truth labels.
• Outcome: The student achieves accuracy close to the teacher's (e.g., 93%) while being much smaller and faster.

4. Cases Where Different Datasets Are Used

4.1. Transfer Learning or Domain Adaptation

• Scenario: The teacher is pretrained on a large, general dataset (e.g., ImageNet for images, Wikipedia or Common Crawl for text), while the student is trained on a smaller, task-specific or domain-specific dataset.
• Rationale: The teacher, often a large foundation model, learns broad, generalizable features from a diverse dataset. These features can be distilled into a smaller student model tailored to a specific domain or task, even if the target dataset is limited.
• Example: In NLP, a teacher like BERT is pretrained on a massive text corpus (e.g., Wikipedia + BookCorpus). The student (e.g., DistilBERT) is trained on a smaller, task-specific dataset (e.g., sentiment analysis on IMDb reviews), using the teacher's outputs to guide learning.
• Benefits: Leverages the teacher's general knowledge to improve student performance on specialized tasks with limited data.
• Challenges: The teacher's knowledge may not fully align with the target domain, requiring careful tuning of the distillation process.

4.2. Semi-Supervised Learning with Unlabeled Data

• Scenario: The teacher is trained on a labeled dataset, while the student is trained on a combination of the same labeled dataset and additional unlabeled data, using the teacher's predictions (pseudo-labels) for the unlabeled portion.
• Rationale: The teacher provides high-quality pseudo-labels for unlabeled data, effectively expanding the student's training set. Particularly useful when labeled data is scarce but unlabeled data is abundant.
• Example: In image classification, the teacher is trained on a labeled subset of CIFAR-100. The student is trained on both the labeled subset and a large pool of unlabeled images.
• Benefits: Improves student performance by leveraging unlabeled data.
• Challenges: The teacher's pseudo-labels may contain errors, especially for out-of-distribution unlabeled data; confidence thresholding or consistency regularization may be needed.

4.3. Privacy-Preserving or Data-Constrained Settings

• Scenario: The teacher is trained on a sensitive or proprietary dataset that cannot be shared, while the student is trained on a different, publicly available, or synthetic dataset.
• Rationale: Knowledge distillation allows the teacher's knowledge to be transferred without directly exposing the original training data.
• Example: A teacher model trained on private medical records is used to distill knowledge into a student model trained on a synthetic or public medical dataset.
• Benefits: Enables knowledge transfer in settings with strict data privacy or access restrictions.
• Challenges: The proxy dataset must be sufficiently similar to the teacher's dataset to ensure effective distillation.

4.4. Data Augmentation or Perturbed Datasets

• Scenario: The teacher is trained on the original dataset, while the student is trained on an augmented or perturbed version (e.g., with added noise, transformations, or adversarial examples).
• Rationale: Training the student on augmented data can improve its robustness or generalization, while the teacher provides stable, high-quality targets.
• Example: In computer vision, the teacher is trained on clean ImageNet images, while the student is trained on ImageNet with random augmentations (rotations, flips, color jitter).
• Benefits: Enhances student robustness to variations in input data.
• Challenges: The augmentations must be carefully chosen to avoid introducing irrelevant or harmful noise.

4.5. Cross-Modal or Multi-Task Distillation

• Scenario: The teacher is trained on one modality or task, while the student is trained on a different modality or task, using shared or aligned knowledge.
• Rationale: The teacher's knowledge from one domain or modality can guide the student in a related but different domain, especially when the tasks share underlying patterns.
• Example: In vision-language models, a teacher trained on a large image-caption dataset distills knowledge into a student trained on a text-only or image-only dataset for a specific task.
• Benefits: Enables knowledge transfer across modalities or tasks.
• Challenges: Requires alignment between the teacher's and student's output spaces, often necessitating additional mapping layers or loss functions.

4.6. Continual Learning or Evolving Data Distributions

• Scenario: The teacher is trained on an initial dataset, while the student is trained on a new dataset that reflects a shifted or updated data distribution.
• Rationale: The teacher provides a stable knowledge base from the original data, while the student adapts to the new distribution.
• Example: In a recommendation system, the teacher is trained on historical user interaction data, while the student is trained on recent data with new user behaviors or items.
• Benefits: Supports adaptation to changing environments while preserving prior knowledge.
• Challenges: The teacher's knowledge may become outdated, so techniques like fine-tuning the teacher or using an ensemble of teachers may be needed.

Considerations for Using Different Datasets

1. Domain Alignment: The datasets should be sufficiently related to ensure the teacher's knowledge is relevant to the student's task. Techniques like feature alignment or adversarial training can help bridge domain differences.
2. Teacher Output Quality: The teacher's outputs must be reliable for the student's dataset. Validate the teacher's performance on a subset of the student's dataset or use confidence-based filtering to discard low-quality predictions.
3. Distillation Loss Design: The distillation loss should account for potential differences in data distributions. Weighting the distillation loss lower than the task-specific loss may help when the teacher's outputs are less reliable.
4. Data Availability and Privacy: Ensure the student's dataset is sufficient for training, and in privacy-sensitive cases, verify that the distillation process complies with data regulations (e.g., GDPR or HIPAA).
5. Evaluation and Validation: Monitor the student's performance on a validation set from its target dataset and compare against a baseline trained without distillation.

When to Avoid Different Datasets

Using different datasets can introduce complexity and risks. Avoid this approach if:

• The datasets are unrelated or have significant domain gaps, and fine-tuning the teacher isn't feasible.
• The student's dataset is large and high-quality, making distillation from a differently trained teacher unnecessary.
• The teacher's performance on the student's data distribution is poor, and there's no way to validate or improve it.

5. Self-Training

What is Self-Training?

Self-training is a semi-supervised learning approach where a model (often called the teacher) is first trained on a labeled dataset, then used to generate pseudo-labels for an unlabeled dataset. These pseudo-labels are combined with the labeled data to train a new model (often called the student), which may be the same or a different architecture. The process can be iterative, with the student becoming the teacher in subsequent rounds.

How Self-Training Works

1. Train the Teacher: Train a model on a labeled dataset $D_L = \{(x_i, y_i)\}$, where $x_i$ are inputs and $y_i$ are ground-truth labels.
2. Generate Pseudo-Labels: Use the trained teacher to predict labels (or probabilities) for an unlabeled dataset $D_U = \{(x_j)\}$. These predictions are called pseudo-labels, denoted $\hat{y}_j$.
3. Select High-Confidence Pseudo-Labels: Often, only pseudo-labels with high confidence (e.g., above a threshold) are used to filter out noisy predictions.
4. Train the Student: Train a new model on the combined dataset $D_L \cup \{(x_j, \hat{y}_j)\}$, where the student learns from both ground-truth labels and pseudo-labels.
5. Iterate (Optional): The student becomes the teacher, generates new pseudo-labels, and the process repeats for multiple iterations.

How Self-Training Improves Model Performance

• Leverages Unlabeled Data: Self-training makes use of abundant unlabeled data, increasing the effective training set size and helping the model learn more robust features.
• Regularization Effect: Pseudo-labels provide a form of regularization, encouraging the model to produce consistent predictions across labeled and unlabeled data.
• Improved Generalization: By exposing the model to a larger and more diverse dataset, self-training can improve performance on out-of-distribution or challenging examples.
• Iterative Refinement: In iterative self-training, each round refines the pseudo-labels, potentially improving their quality.

Challenges

• Error Propagation: If the teacher's pseudo-labels are incorrect, the student may learn from noisy or biased labels. Confidence thresholding or soft labels (probabilities) can mitigate this.
• Limited Teacher Quality: If the initial teacher is trained on a small labeled dataset, its pseudo-labels may not be reliable.
• Domain Shift: If the unlabeled data differs significantly from the labeled data, pseudo-labels may not align with the target task.

6. Noisy Student Training

What is Noisy Student Training?

Noisy Student Training (Xie et al., 2020) is an advanced self-training method that incorporates noise and regularization to improve the student model's performance beyond the teacher's. Unlike standard self-training, Noisy Student explicitly adds stochastic elements (e.g., data augmentation, dropout) to the student's training process to make it more robust and prevent it from simply memorizing the teacher's predictions.

How Noisy Student Training Works

1. Train the Teacher: Train a teacher model on a labeled dataset $D_L$, similar to self-training.
2. Generate Pseudo-Labels: Use the teacher to predict pseudo-labels for a large unlabeled dataset $D_U$. Hard labels (class predictions) are often used instead of soft labels to simplify the process.
3. Train the Student with Noise: Train a student model (typically larger or equal in capacity to the teacher) on $D_L \cup \{(x_j, \hat{y}_j)\}$, but introduce noise and regularization during training:
   • Data Augmentation: Apply strong augmentations (e.g., RandAugment for images) to both labeled and unlabeled inputs.
   • Model Noise: Use techniques like dropout, stochastic depth, or random layer freezing in the student model.
   • Balanced Dataset: Ensure the labeled and unlabeled data are balanced.
4. Iterate: Use the student as the new teacher, generate updated pseudo-labels, and repeat. Each iteration typically improves the model's performance.

Key Differences from Self-Training

• Noise Injection: Noisy Student deliberately adds noise to prevent overfitting to the teacher's pseudo-labels and encourage robustness.
• Equal or Larger Student: Unlike knowledge distillation where the student is smaller, Noisy Student often uses a student of equal or greater capacity than the teacher.
• Iterative Improvement: Noisy Student emphasizes multiple iterations, with each student potentially outperforming the previous teacher.

How Noisy Student Improves Model Performance

• Robustness through Noise: The added noise forces the student to learn more generalizable features, preventing overfitting to the teacher's predictions or biases in the pseudo-labels.
• Scales with Unlabeled Data: Noisy Student leverages large amounts of unlabeled data, which is critical for improving performance in data-scarce settings.
• Iterative Refinement: Each iteration refines the pseudo-labels, as the student-turned-teacher becomes more accurate, leading to a virtuous cycle of improvement.
• Surpassing the Teacher: By combining noise, a larger student model, and iterative training, Noisy Student can produce models that outperform the initial teacher.

Example

In the Noisy Student paper, an EfficientNet teacher was trained on the labeled ImageNet dataset. It generated pseudo-labels for 300M unlabeled images from JFT-300M. A larger EfficientNet student was trained on the combined dataset with strong augmentations (RandAugment) and dropout, achieving higher accuracy than the teacher. After multiple iterations, the final model set new benchmarks on ImageNet.

Challenges

• Computational Cost: Noisy Student requires significant computational resources, as it involves training large models and iterating multiple times over large datasets.
• Noise Tuning: The type and strength of noise (augmentations, dropout) must be carefully tuned to balance robustness and learning stability.
• Pseudo-Label Quality: As with self-training, poor pseudo-labels can harm performance, especially in early iterations.

7. Connection to Knowledge Distillation and Different Datasets

Both self-training and Noisy Student are closely related to knowledge distillation, particularly in scenarios where the teacher and student are trained on different datasets (e.g., labeled vs. unlabeled data):

• Different Datasets: In both methods, the teacher is typically trained on a labeled dataset, while the student is trained on a combination of the labeled dataset and an unlabeled dataset (or a different dataset with pseudo-labels).
• Knowledge Transfer: Like knowledge distillation, self-training and Noisy Student transfer knowledge from the teacher to the student via pseudo-labels (hard or soft).
• Improving Performance: These methods are particularly effective when the labeled dataset is small but the unlabeled data is abundant.

How These Techniques Improve Model Performance

1. Increased Data Utilization: Both methods leverage unlabeled data, which is often plentiful, to expand the training set. This exposes the model to more diverse patterns, improving generalization.
2. Regularization: Pseudo-labels and noise (in Noisy Student) act as regularizers, preventing overfitting to the labeled data.
3. Iterative Improvement: Iterative training refines pseudo-labels, leading to progressively better models.
4. Handling Limited Labeled Data: In settings with few labeled examples, these methods use unlabeled data to compensate, achieving performance close to fully supervised models.
5. Domain Adaptation: When the unlabeled data comes from a slightly different distribution, these methods can help the student adapt to the target domain.

Practical Considerations

• When to Use Self-Training: Self-training is effective for small to medium unlabeled datasets and is used for simpler semi-supervised tasks with limited compute resources or when the student model is similar to the teacher.
• When to Use Noisy Student: Use Noisy Student when you have access to large unlabeled datasets, sufficient compute, and want to maximize performance, potentially surpassing the teacher.
• Implementation Tips:
  • Confidence Thresholding: Filter pseudo-labels by confidence (e.g., keep predictions with probability > 0.9) to reduce noise.
  • Strong Teacher: Start with a high-quality teacher (e.g., pretrained on a large dataset) to ensure reliable pseudo-labels.
  • Noise Tuning: In Noisy Student, experiment with augmentation policies (e.g., RandAugment) and regularization strength.
  • Iteration Strategy: Limit iterations to avoid error accumulation, and monitor validation performance to stop early if needed.

Benchmark (Up-to-Date)

Public Benchmark

Disclaimer:

• This is just a benchmark on some standard datasets, which means the model does not necessarily become the actual "SOTA" for in-the-wild applications.
• This benchmark focuses on accuracy metrics but no latency metrics, which are also an essential part of the real-time applications.

Personal Notes

Prominent Works

These works may not be "SOTA" on some specific datasets but have high potential for real-world, in-the-wild applications.

Datasets & Simulator

Other References

Perspective Transform

The homography (perspective transform) is a geometric transformation relating 2 different planes. Straight lines remain straight after transformation.

Each point $(x, y)$ on plane $\pi$ maps to $(x', y')$ on plane $\pi'$:

where $s$ is a scale factor and $\mathbf{H}$ is a 3×3 homography matrix with 8 DoF. To compute $\mathbf{H}$ you need 4 point correspondences, with no 3 of them collinear.

---

Affine Transformation

Definition

An affine transformation preserves lines and parallelism, but not necessarily Euclidean distances and angles:

• Points on the same line remain collinear after transformation.
• Parallel lines stay parallel after transformation.
• The ratio of any pair of segments is preserved; midpoints remain midpoints.

Examples

• Translation: moving a figure without changing orientation or size.
• Rotation: turning a figure around a point.
• Scaling: enlarging or shrinking a figure.
• Shearing: skewing a figure.

Equation

The problem

Standard object detectors are trained on images resized to a fixed input resolution (e.g. 640×640). When the source images are high-resolution and objects of interest are small or far from the camera — a distant pedestrian, a small container code, a worker near the edge of a railway platform — that resize step shrinks them down to just a handful of pixels. Most of the training signal ends up dominated by large, easy objects, and recall on the small ones suffers.

---

SAHI in a nutshell

SAHI (Slicing Aided Hyper Inference) tackles this at inference time: instead of feeding the detector one downscaled image, it slices the full-resolution image into overlapping tiles, runs detection independently on each tile at full resolution, then merges all tile-level detections back into original image coordinates with NMS. Small objects that would've been lost to downscaling now occupy a much larger fraction of each tile, so the detector actually has pixels to work with.

---

Adapting the idea for training data instead

Running multiple slice passes at inference is expensive — it multiplies the number of forward passes per image, which conflicts with a real-time deployment budget. So rather than slicing at inference, I moved the same idea earlier in the pipeline: use SAHI-style slicing to build a better training set, and keep inference itself as a single full-image forward pass.

At a high level, the preprocessing step:

1. Slices each training image into overlapping tiles (with a configurable tile size and overlap ratio).
2. Remaps ground-truth boxes into tile coordinates, keeping boxes above a visibility/IoU threshold and dropping or clipping the rest.
3. Oversamples tiles that contain small-object instances, so the training distribution isn't dominated by easy, large-object crops.
4. Mixes the sliced crops back in with the original full-resolution images, so the model still sees global scene context and doesn't overfit to tile-scale statistics.

Paired with YOLOv8-p2 and YOLOv9n — both already extended with a higher-resolution P2 detection head for small-object sensitivity — this preprocessing step gave a meaningful recall boost on small, distant objects, while inference stayed a single full-image pass, keeping real-time latency intact.

---

Illustrative sketch

A simplified sketch of the slicing + remapping step (illustrative, not the production implementation):

def slice_image(image, boxes, tile_size=640, overlap=0.2, min_visibility=0.3):
    """Slice an image into overlapping tiles, remapping ground-truth boxes."""
    stride = int(tile_size * (1 - overlap))
    tiles = []

    for y in range(0, image.height, stride):
        for x in range(0, image.width, stride):
            tile_box = (x, y, x + tile_size, y + tile_size)
            tile_img = image.crop(tile_box)

            tile_boxes = []
            for box in boxes:
                clipped = clip_box(box, tile_box)
                if visibility(clipped, box) >= min_visibility:
                    tile_boxes.append(to_tile_coords(clipped, tile_box))

            tiles.append((tile_img, tile_boxes))

    return tiles

Reference: Akyon et al., “Slicing Aided Hyper Inference and Fine-tuning for Small Object Detection” (2022).