Skip to main content

Working with Frames

View and edit SingleFrames in the Dataset page. After selecting a Hyper-Dataset version, the Version Browser shows a sample of frames and enables viewing SingleFrames and FramesGroups, and edit SingleFrames, in the frame viewer. Before opening the frame viewer, you can filter the frames by applying simple or advanced filtering logic.

Dataset page

Frame Viewer#

Frame viewer allows you to view and edit annotations which can be FrameGroup objects (Regions of Interest) and FrameGroup labels applied to the entire frame not a region of the frame, the frame details (see frames), frame metadata, the raw data source URI, as well as providing navigation and viewing tools.

Frame viewer

Frame Viewer Controls#

Use frame viewer controls to navigate between frames in a Hyper-Dataset Version, and control frame changes and viewing.

Control IconActionsShortcut
Jump backwards iconJumps backwards ten framesCTRL + Left arrow
Jump to previous unfiltered annotationGo to the previous frame containing a non-filtered annotation. The filter is the minimum confidence level setting. If the confidence level filter is set to zero, any frame containing annotations matches the filter.Not applicable
Previous frame iconGo to the previous frameLeft arrow
Next frame iconGo to the next frameRight arrow
Jump to next unfiltered annotationGo to the next frame containing a non-filtered annotation (same filter as Jump to previous unfiltered annotation).Not applicable
Jump forwards iconJump forwards ten framesCTRL + Right arrow
Reload frame iconReload the frame.Not applicable
Undo iconUndo changes.Ctrl + Z
Redo iconRedo changes.Ctrl + Y
Autofit iconAutofitNot applicable
Zoom in iconZoom in+ or Ctrl + Mouse wheel
Zoom out iconZoom out- or Ctrl + Mouse wheel
Percentage textboxZoom percentageNot applicable

Additional keyboard shortcuts#

General Controls

Hold Spacebar + Press and hold image + DragMove around image. NOTE: If using a touchpad, this only works if the Disable touchpad while typing setting is turned off
EscEscape frame viewer and return to dataset page

General Annotation Controls

DeleteRemove annotation
Alt + ] / [Choose a default ROI label by navigating between previous / next labels
Shift + MEdit metadata
Shift + TabPrevious annotation (use after clicking an annotation)

Mode-specific annotation controls

Hold ShiftKey points (Key points mode)While holding Shift, add new points to the ROI by left clicking, and move a single point by dragging it with your mouse
EnterKey points (Key points mode)Complete annotation
EscKey points (Key points mode), Polygon (Polygon mode)Cancel annotation process

Viewing and Editing Frames#

To view / edit a frame in the frame editor

  1. Locate your frame by applying a simple frame filter or advanced frame filter, and clicking LOAD MORE, if required.
  2. Click the frame thumbnail. The frame editor appears.
  3. Do any of the following:
    • View frame details, including:
      • Frame file path
      • Dimensions of the image or video
      • Frame details
      • Frame metadata
      • Annotations
        • Frame objects - Labeled Regions of Interest, with confidence levels and custom metadata per frame object.
        • Frame labels - Labels applied to the entire frame, not a region in the frame.
    • Optionally, filter annotations by confidence level using the Minimum confidence slider.
    • Add, change, and delete annotations and frame metadata.

To save frames changes at any time, click SAVE (below the annotation list area).

Viewing FrameGroups#

Viewing and editing frames in a FrameGroup is similar to viewing and editing SingleFrames. Click the FrameGroup in the Hyper-Dataset. In the frame viewer, select SingleFrame to view / modify from a dropdown list in the Current Source section.

Frame dropdown menu in FrameGroup

Filtering Frames#

Simple Frame Filtering#

Simple frame filtering applies one annotation object (ROI) label and returns frames containing at least one annotation with that label.

To apply a simple frame filter:

  • In the Version Browser, choose a label on the label list.

For example:

  • Before filtering, the Version Browser in the image below contains seven frames.
View a screenshot

Unfiltered version browser

  • A simple label filter for person shows three frames with each containing at least one ROI labeled person.
View a screenshot

Filtered version browser

Advanced Frame Filtering#

Advanced frame filtering applies sophisticated filtering logic, which is composed of as many frame filters as needed, where each frame filter can be a combination of ROI, frame, and source rules.

  • ROI rules use include and exclude logic to match frames by ROI label; an ROI label can match frames containing at least one annotation object (ROI) with all labels in the rule.
  • Frame rules and source rules use Lucene queries with AND, OR, and NOT logic. Frame rules apply to frame metadata.
  • Source rules apply to frame source information.

To apply advanced filters:

  1. In the Version Browser, click Switch to advanced filters.

  2. In a FRAME FILTER, create one of the following rules:

    • ROI rule

      • Choose Include or Exclude, select ROI labels, and optionally set the confidence level range.

      • To switch from the ROI dropdown list to a Lucene query mode, click .

    • Frame rule - Enter a Lucene query using frame metadata fields in the format meta.<key>:<value>.

    • Source rule - Enter a Lucene query using frame metadata fields in the format sources.<key>:<value>.


ROI Rules#

  • Create one ROI rule for person shows the same three frames as the simple frame filter (above).

    View a screenshot

    Adding an ROI rule

  • In the ROI rule, add a second label. Add partially_occluded. Only frames containing at least one ROI labeled as both person and partially_occluded match the filter.

    View a screenshot

    Add label to ROI rule

    By opening a frame in the frame viewer, you can see an ROI labeled with both.

    View a screenshot

    Labeled ROIs in frame viewer

Frame Rules#

Filter by metadata using Lucene queries.

  • Add a frame rule to filter by the metadata key dangerous for the value of no.

    View a screenshot

    Filter by metadata

    By opening a frame in the frame viewer, you can see the metadata.

    View a screenshot

    Frame metadata in frame viewer

Source Rules#

Filter by sources using Lucene queries.

  • Add a source rule to filter for sources URIs with a wildcards.

    View a screenshot

    Filter by source

Use Lucene queries in ROI label filters and frame rules.


Use the MASKS panel to select a mask or multiple masks to apply over the frame.

To view / hide a specific mask, click Eye Show. In order to view all masks, click Show all / Hide all.

To adjust the transparency of a mask, use its opacity slider.


The Active mask labels section displays the color mapping of the mask labels. The panel presents labels only from masks that are currently displayed.

To modify a label's color and opacity:

  1. Click the colored circle next to the label
  2. Select a new color
  3. Adjust the opacity slider
  4. Click OK


Frame Objects (Regions of Interest)#

You can add annotations by drawing new bounding areas, and copying existing annotations in the same or other frames.

To draw a bounding area for a new annotation:

  1. Optionally, select a default label in the Default ROI Label(s) list. New annotations are automatically assigned this label.

  2. Click one of the following modes and create a bounding area in the frame:

    • Rectangle mode icon - Rectangle mode: Drag a rectangle onto the frame.
    • Ellipse mode icon - Ellipse mode: Drag an ellipse onto the frame.
    • Polygon mode icon - Polygon mode: Click the polygon vertices onto the frame.
    • Key points mode icon - Key points mode: Click each keypoint onto the frame. After the clicking the last keypoint, click the first again to close the bounding area.

      A new annotation is created.

  3. In the newly created annotation, select or type a label.

  4. Optionally, add metadata. This is metadata for the annotation, not the entire frame.

  5. Optionally, lock the annotation.

  6. If you move to another frame, the frame editor automatically saves changes. Otherwise, if you exit the frame editor, you are prompted to save.

To copy an annotation:

  1. Click the annotation or bounded area in the image or video clip.
  2. In the annotation, click Copy annotation icon (copy annotation)
  3. Optionally, navigate to a different frame.
  4. Click PASTE. The new annotation appears in the same location as the one you copied.
  5. Optionally, to paste the same annotation, click PASTE again in the desired frame.

To copy all annotations:

  1. Click COPY ALL.
  2. Optionally, navigate to a different frame.
  3. Click PASTE.

Frame Labels#

To add frame labels:

  1. Expand the FRAME LABELS area.
  2. Click + Add new.
  3. Enter a label.
  4. Optionally, click .

Annotation Management#

To move annotations:

  • Move a bounded area by clicking on it and dragging.

To resize annotations:

  • Resize a bounded area by clicking on a vertex and dragging.

To modify annotation metadata:

  • In the label, click edit and then in the popup modify the metadata dictionary (in JSON format).

To modify annotation colors:

  • Modify the color of an area by clicking the circle in the label name and select a new color.

To lock annotations:

  • All annotations - Above the annotations, click the lock / unlock.
  • A specific annotation - In an annotation, click the lock / unlock.

To delete annotations:

  1. Click the annotation or bounded area in the image or video clip.
  2. Press the DELETE key, or in the annotation, click X.

To add, change, or delete labels to annotations labels:

  • Add - Click in the annotation and choose a label from the label list, or type a new label.
  • Change - In the annotation label textbox, choose a label from the list or type a new label.
  • Delete - In the annotation, in the label area, click the label's X.

Frame Metadata#

To edit frame metadata:

  • Expand the FRAME METADATA area, click edit, and then in the popup modify the metadata dictionary (in JSON format).