YOLOv8 How to Train Custom Data

To train YOLOv8 on custom data, you prepare your images and labels in YOLO format, create a data.yaml file, and run the yolo train command with your dataset path. This article walks through the entire process, from dataset setup to evaluating the trained model, so you can detect your own objects.

What Is YOLOv8?

YOLOv8 (You Only Look Once version 8) is a state-of-the-art object detection model developed by Ultralytics. Unlike older two-stage detectors, YOLO treats detection as a single regression problem, predicting bounding boxes and class probabilities directly from full images in one pass. This makes it extremely fast while maintaining high accuracy. YOLOv8 supports object detection, instance segmentation, and image classification, and it is designed to be easy to use with a simple Python API and command-line interface.

The key innovation in YOLOv8 is its anchor-free detection head, which simplifies the model architecture and improves generalization. It also uses a CSPDarknet backbone and a PANet neck for feature fusion. For most users, the practical advantage is that training custom models requires only a few lines of code and a properly formatted dataset.

Preparing Your Custom Dataset

Before training, you need a dataset of images and corresponding labels. The images can be in any common format (JPG, PNG, etc.). Labels must be in YOLO format: a plain text file for each image, with the same base name as the image, placed in a labels folder. Each line in the label file represents one object and contains five numbers:

• Class ID: an integer starting from 0 (e.g., 0 for “cat”, 1 for “dog”)
• Center X: normalized x-coordinate of the bounding box center (0 to 1)
• Center Y: normalized y-coordinate of the bounding box center (0 to 1)
• Width: normalized width of the bounding box (0 to 1)
• Height: normalized height of the bounding box (0 to 1)

For example, a label file for an image with one cat centered at (0.5, 0.5) with width 0.3 and height 0.4 would contain: 0 0.5 0.5 0.3 0.4. You can create these labels manually, but it is much easier to use a labeling tool like LabelImg or Roboflow, which can export directly in YOLO format.

Organize your data into three folders: train, val, and optionally test. Each folder should contain an images subfolder and a labels subfolder. A typical structure looks like this:

dataset/
  train/
    images/
    labels/
  val/
    images/
    labels/

The more diverse and representative your training images are, the better your model will generalize. Aim for at least 1,500 instances per class for good accuracy, though smaller datasets can still work for simple tasks.

Creating the data.yaml File

YOLOv8 needs a configuration file that tells it where your data is and what classes you have. Create a file named data.yaml (any name works) with the following content:

train: dataset/train/images
val: dataset/val/images

nc: 2
names: ['cat', 'dog']

• train and val: absolute or relative paths to your image folders
• nc: number of classes
• names: list of class names in order of their IDs

If you want to use a test set, add a test key. The paths can be absolute or relative to where you run the training command. Make sure the paths are correct, as YOLOv8 will throw an error if it cannot find the images.

Training the Model

With your dataset ready, you can start training. Install the Ultralytics package if you have not already:

pip install ultralytics

Then run the training command from the terminal:

yolo train data=data.yaml model=yolov8n.pt epochs=100 imgsz=640

Here is what each argument means:

• data: path to your data.yaml file
• model: base model to start from. Use a pre-trained model like yolov8n.pt (nano), yolov8s.pt (small), yolov8m.pt (medium), yolov8l.pt (large), or yolov8x.pt (extra-large). Starting from a pre-trained model is called transfer learning and dramatically speeds up training and improves accuracy.
• epochs: number of full passes over the dataset. More epochs generally improve accuracy but increase training time.
• imgsz: input image size in pixels. 640 is a good default; larger sizes capture more detail but use more memory.

You can also train from scratch by using model=yolov8n.yaml, but that requires much more data and time. For most custom projects, starting from a pre-trained model is the best choice.

During training, YOLOv8 will display progress bars, loss values, and metrics like mAP (mean Average Precision). The model and training results are saved in a folder named runs/detect/train (or runs/segment/train for segmentation).

Evaluating and Using Your Model

After training, you can evaluate the model on your validation set using:

yolo val model=runs/detect/train/weights/best.pt data=data.yaml

This will output metrics such as precision, recall, and mAP50-95. The best.pt file is the model with the highest validation accuracy during training; last.pt is the final epoch.

To run inference on new images, use:

yolo predict model=runs/detect/train/weights/best.pt source='path/to/image.jpg'

You can also use the Python API for more control:

from ultralytics import YOLO
model = YOLO('runs/detect/train/weights/best.pt')
results = model('image.jpg')
results[0].show()

For real-time video detection, pass a video file or webcam stream as the source.

Common Problems and Solutions

Here are frequent issues you might encounter and how to fix them:

• Low accuracy: If your mAP is low, add more training images, especially of objects in different orientations and lighting. Also check for label errors and class imbalance.
• Overfitting: If training loss decreases but validation loss increases, your model is memorizing the training data. Try adding data augmentation (YOLOv8 does this automatically), increasing the dataset size, or using a smaller model.
• Out of memory: Reduce the batch size (default is 16) or image size. You can set batch=8 or imgsz=416 to fit in GPU memory.
• Wrong paths: Double-check that the paths in data.yaml are correct and that the images and labels are in the right folders. YOLOv8 will raise a clear error if files are missing.
• Class ID mismatch: The class IDs in your label files must match the order in names. If you have 3 classes, IDs should be 0, 1, 2.

Frequently Asked Questions

How many images do I need to train YOLOv8?
There is no fixed number, but a good rule of thumb is at least 1,500 instances per class for a robust model. For simple tasks with one or two classes, a few hundred images can work if they are diverse. Quality matters more than quantity; ensure objects are clearly visible and labeled correctly.

Can I train YOLOv8 on CPU only?
Yes, but it will be very slow. Training a small model like YOLOv8n on a CPU can take hours or days depending on dataset size. For serious training, use a GPU with at least 8GB VRAM. You can also use Google Colab’s free GPU for small projects.

What is the difference between YOLOv8n and YOLOv8x?
These are different model sizes. YOLOv8n (nano) is the smallest and fastest, with around 3.2 million parameters, while YOLOv8x (extra-large) has about 68 million parameters and is slower but more accurate. Choose a size based on your speed and accuracy needs; start with nano or small for prototyping.

Should I train from scratch or use a pre-trained model?
Always use a pre-trained model (e.g., yolov8n.pt) unless you have a massive dataset (tens of thousands of images per class) and a good reason. Transfer learning from COCO weights gives you a head start and typically results in better accuracy with less data and time.

How do I export my trained model to other formats?
Use the export command: yolo export model=best.pt format=onnx for ONNX, or format=tflite for TensorFlow Lite, and format=coreml for Apple devices. This allows you to deploy your model on mobile, edge devices, or other frameworks.

Why is my model detecting only one class even though I have multiple?
This usually indicates a class imbalance or a labeling issue. Check that your label files contain the correct class IDs and that each class has a sufficient number of examples. Also verify that the nc value in data.yaml matches the actual number of classes.

Related

• Summer Camp Tech Essentials: Top Picks
• EU’s Once-Only Principle: What It Means
• Sleep a Mac from the Command Line

in t | Finish time | min | 50 | 0
in d | Distance | km | 10 | 0 | 100000
out pace | Average pace | min/km | t / d | 2
out speed | Average speed | km/h | d / (t / 60) | 2