For this quick start, we'll be using OpenCV's Computer Vision Annotation Tool (CVAT). You will be able to use an existing model to pre-annotate your images or videos and then continuously train and improve your model on new data.
In order to use CVAT on Onepanel, we need to create a new Workspace for CVAT. More information on Workspaces can be found here.
We will be using following Workspace Template to create the Workspace:
A copy of this template is readily available in "Template Builder"
Here, we used Docker images for CVAT to create the Workspaces and exposed few ports for CVAT's use.
Creating super user
CVAT requires super user permission to perform certain tasks. Onepanel automatically creates a superuser when you execute CVAT workflow. You can set
password via environment variables shown below. If you don't specify those variables, the default
password will be
Setting up environment variables
In order to use several features of CVAT such as training an annotaiton model we need to set some environment variables. You can easily set environment variables by clicking Settings button on top nav bar. This will open up a settings page, where you can set environment variables. Following is an example of setting an environment variable.
You need to set following environment variables:
- AWS_BUCKET_NAME: Bucket you want to store your data into. Here, data refers to the dataset that will be used for training models.
- AWS_ACCESS_KEY_ID: AWS access key for training new annotation models.
- AWS_SECRET_ACCESS_KEY: AWS secret access key for training new annotation models.
- AWS_S3_PREFIX: Directory in
AWS_BUCKET_NAMEwhere all the data will be stored. Here, data refers to the dataset that will be used for training models.
- ONEPANEL_OD_TEMPLATE_ID: Template ID for Tensorflow Object Detection. Required only if you are training a new annotation model.
- ONEPANEL_MASKRCNN_TEMPLATE_ID: Template ID for MaskRCNN Segmentation. Required only if you are training a new annotation model.
- ONEPANEL_AUTHORIZATION: Token/password for Onepanel login.
- DJANGO_SUPERUSER_USERNAME: Username for the superuser. Default is
- DJANGO_SUPERUSER_PASSWORD: Password for the superuser. Default is
- SYNC_S3_BUCKET_NAME: Bucket to sync shared storage (
sharedrive will be used for the same) with. You will be able to use data available on this bucket while creating new tasks and models.
- SYNC_S3_PREFIX: Prefix (directory) to use for above bucket. By default its empty. Thus, all data in that bucket will be fetched.
- SYNC_DIRECTION: Direction for SYNC. Options:
down: One way sync. Downloads all data from s3 to local folder.
up: One way sync. Uploads all data from local folder to S3. If you delete files from local folder it will delete those files from S3 as well.
both: Two way sync. Syncs files in both direction.
- SYNC_DELAY: Specifies time in seconds for the syncer to check for file updates. Default is
up will delete files from cloud storage if you delete files from local storage.
Creating new tasks
Once you're inside CVAT dashboard, you can create new tasks to start annotating. You will find a Create New Task button on top, clicking on it will open up a new pop up window as follows:
Now, you give this task a name you like. Then, labels that you are interested in annotating (i.e car, bicycle). You also need to select the source of your data (images). You can upload from your local machine or use data uploaded to S3.
Using data from S3
If you want to use data from your S3 drive while creating new task. You must set
SYNC_S3_BUCKET_NAME environment as mentioned above. Now, you can select
Connected file share while creating new task. It will show all the files from S3 bucket, or from the directory if you set
Once you have created a new task, you can start annotating your data. CVAT supports points, box, polylines, polygons for annotation. So, the first thing you should do is go to left sidebar and select the type of annotation you want as shown below.
If you want to annotate points, then select Points instead of Box which is a default choice. Once you select points, you can start annotating by clicking on Create Shape, clicking on image where you want to put the point and then click on stop shape. Or alternatively you can use keyboard shortcut N instead of Create Shape/Stop Shape. Make sure you periodically save your annotation by pressing ctrl + s.
The same process follows for bounding boxes. Select Box and press N to start annotating once done, press N to finish annotation. Sometimes N will draw
rectangle shape and sometimes it will draw
rectangle track. In order to ensure you have the right one, please draw one box by manually selecting Shape or Track from the UI. Then, onward pressing N will draw what you selected for the box.
If you want to change the class of an object. Finish drawing bounding box around an object, then go to right sidebar and change the class from a dropdown menu. The same has been heighted in blue color in above picture.
Similarly, select polygons or polylines and follow same procedure for annotation.
It is highly recommended that you dump your annotation periodically. In case of any failure, this can be used to recover the tasks.
Using pre-annotation model
Onepanel’s CVAT supports a feature to pre-annotate images for common objects. In order to use any pre-annotation feature, you first need to upload the model. By default, we provide a default model for bounding box annotation. Click on Models, and give a name to it. Click on select files and upload your model (.pb and .csv file). Hit submit to upload the model.
Once you submit your model, click on Models again and you will find your model in the list. You can also use files from
Connected file share if you set appropriate environment variables. See
Setting up environment variables section.
Once you have the models in Model Manager. Click on Automatic Annotation under Actions menu. Then, you will be asked to select the model you want to use for pre-annotation. You can also control the class mapping from your task’s classes to model’s classes. Once done, click on Submit to start pre-annotation. Once it's done, you can click on the task link to access the annotation.
Training new annotation model
Onepanel also allows you to further finetune your model for annotation. Once you are done with your annotation or adjustment to pre-annotation, you can train a new model on it. To do so, go to dashboard and click on Actions under the task for which you want to train a model.
There, you can select the Tensorflow OD API for bounding boxes or Mask RCNN for segmentation.
For TensorFlow OD API, we support multiple models. In fact, its dynamic. You can also train the model you like as long as it is supported by Tensorflow Object Detection API.
You can also select the base model to finetune on top of. The list of base models are models that you trained previously. This is optional. By default, a model trained on COCO will be used. For more information, see section on
Create annotation model.
How to choose the model:
If you are unsure about which model to use, we usually suggest ssd-mobilenet-v2 since ssd-based models are faster and accurate enough for most of the work. Faster-rcnn (frcnn) models are more accurate in general but they will be relatively slow during training as well inference. If accuracy is more important to you, we suggest you go with frcnn-res50-coco model.
How to run inference on test data using trained model
Often you are required to make prediction on test data. Using CVAT on Onepanel, you can easily train/test your model and visualize output. Once you have the trained model, upload it to the CVAT by clicking on
Create new model on
Now, create a task with your test data.
Click on Actions for that task and select Automatic annotation. Select the model you just uploaded and hit submit. It will run the inference using the model you selected. Below is a sample frame whose output was generated using the trained model.