tessl/pypi-oemer

End-to-end Optical Music Recognition (OMR) system for transcribing musical notation from images into structured MusicXML format.

—

Pending

Overview

Eval results

Files

Staffline Detection and Analysis

Name: tessl/pypi-oemer
Author: tessl

Detection and analysis of musical staff lines, which form the foundation for all subsequent processing steps. Staff lines provide the coordinate system for pitch interpretation and serve as reference points for symbol positioning.

Capabilities

Primary Staff Extraction

Extract staff lines and organize them into complete staff structures.

def extract(splits: int = 8, line_threshold: float = 0.8, horizontal_diff_th: float = 0.1, unit_size_diff_th: float = 0.1, barline_min_degree: int = 75) -> Tuple[ndarray, ndarray]:
    """
    Main staff extraction function that detects and analyzes staff lines.
    
    Parameters:
    - splits (int): Number of horizontal splits for processing (default: 8)
    - line_threshold (float): Threshold for staff line detection (default: 0.8)
    - horizontal_diff_th (float): Threshold for horizontal alignment validation (default: 0.1)
    - unit_size_diff_th (float): Threshold for unit size consistency (default: 0.1)
    - barline_min_degree (int): Minimum angle in degrees for barline detection (default: 75)
    
    Returns:
    Tuple containing:
    - staffs (ndarray): Array of Staff instances
    - zones (ndarray): Array of zone boundaries for each staff group
    
    Raises:
    StafflineException: If staff detection fails or results are inconsistent
    """

def extract_part(pred: ndarray, x_offset: int, line_threshold: float = 0.8) -> List[Staff]:
    """
    Extract staff structures from a specific image region.
    
    Parameters:
    - pred (ndarray): Binary prediction array for the image region
    - x_offset (int): Horizontal offset of this region in the full image
    - line_threshold (float): Detection threshold for staff lines
    
    Returns:
    List[Staff]: List of detected Staff instances in this region
    """

def extract_line(pred: ndarray, x_offset: int, line_threshold: float = 0.8) -> Tuple[ndarray, ndarray]:
    """
    Extract individual staff lines from prediction data.
    
    Parameters:
    - pred (ndarray): Binary prediction array
    - x_offset (int): Horizontal offset in the full image
    - line_threshold (float): Detection threshold
    
    Returns:
    Tuple containing line positions and metadata
    """

Staff Line Representation

Individual staff line with position and geometric properties.

class Line:
    """
    Represents a single staff line with geometric properties.
    
    Attributes:
    - points (List[Tuple[int, int]]): List of (x, y) coordinates on the line
    """
    
    def add_point(self, y: int, x: int) -> None:
        """
        Add a coordinate point to this staff line.
        
        Parameters:
        - y (int): Y coordinate of the point
        - x (int): X coordinate of the point
        """
    
    @property
    def y_center(self) -> float:
        """Get the average Y coordinate of all points on this line."""
    
    @property
    def y_upper(self) -> int:
        """Get the minimum Y coordinate (top) of this line."""
    
    @property
    def y_lower(self) -> int:
        """Get the maximum Y coordinate (bottom) of this line."""
    
    @property
    def x_center(self) -> float:
        """Get the average X coordinate of all points on this line."""
    
    @property
    def x_left(self) -> int:
        """Get the minimum X coordinate (left) of this line."""
    
    @property
    def x_right(self) -> int:
        """Get the maximum X coordinate (right) of this line."""
    
    @property
    def slope(self) -> float:
        """Get the estimated slope of this line."""

Complete Staff Representation

A complete musical staff consisting of five staff lines.

class Staff:
    """
    Represents a complete musical staff with five staff lines.
    
    Attributes:
    - lines (List[Line]): List of the five staff lines (bottom to top)
    - track (int): Track number for multi-staff systems
    - group (int): Group number for staff groupings (e.g., piano grand staff)
    - is_interp (bool): Whether this staff was interpolated from partial data
    """
    
    def add_line(self, line: Line) -> None:
        """
        Add a staff line to this staff.
        
        Parameters:
        - line (Line): The staff line to add
        
        Raises:
        ValueError: If trying to add more than 5 lines
        """
    
    def duplicate(self, x_offset: int = 0, y_offset: int = 0) -> 'Staff':
        """
        Create a duplicate of this staff with optional offset.
        
        Parameters:
        - x_offset (int): Horizontal offset for the duplicate (default: 0)
        - y_offset (int): Vertical offset for the duplicate (default: 0)
        
        Returns:
        Staff: New Staff instance with offset applied
        """
    
    @property
    def y_center(self) -> float:
        """Get the vertical center of this staff (middle of the 5 lines)."""
    
    @property
    def y_upper(self) -> int:
        """Get the top boundary of this staff."""
    
    @property
    def y_lower(self) -> int:
        """Get the bottom boundary of this staff."""
    
    @property
    def x_center(self) -> float:
        """Get the horizontal center of this staff."""
    
    @property
    def x_left(self) -> int:
        """Get the left boundary of this staff."""
    
    @property
    def x_right(self) -> int:
        """Get the right boundary of this staff."""
    
    @property
    def unit_size(self) -> float:
        """
        Get the unit size (average distance between staff lines).
        
        This is the fundamental measurement unit for all musical elements.
        All other measurements (note sizes, stem lengths, etc.) are 
        proportional to this unit size.
        """
    
    @property
    def incomplete(self) -> bool:
        """Check if this staff has fewer than 5 lines."""
    
    @property
    def slope(self) -> float:
        """Get the average slope of all lines in this staff."""

Staff Line Position Enumeration

Enumeration for staff line positions within a staff.

class LineLabel(enum.Enum):
    """
    Enumeration for staff line positions.
    
    Values represent the five staff lines from bottom to top.
    """
    FIRST = 0    # Bottom staff line
    SECOND = 1   # Second staff line from bottom
    THIRD = 2    # Middle staff line
    FOURTH = 3   # Fourth staff line from bottom
    FIFTH = 4    # Top staff line

Usage Examples

Basic Staff Detection

from oemer.staffline_extraction import extract
from oemer.layers import register_layer, get_layer
import numpy as np
import cv2

# Load and prepare image
image = cv2.imread("sheet_music.jpg")
register_layer("original_image", image)

# Simulate staff predictions (in real usage, this comes from neural network)
h, w = image.shape[:2]
staff_pred = np.random.randint(0, 2, (h, w), dtype=np.uint8)
register_layer("staff_pred", staff_pred)

# Extract staff lines
try:
    staffs, zones = extract(
        splits=8,           # Process in 8 horizontal sections
        line_threshold=0.8, # High confidence threshold
        horizontal_diff_th=0.1,  # Strict alignment requirement
        unit_size_diff_th=0.1    # Consistent spacing requirement
    )
    
    print(f"Detected {len(staffs)} staff structures")
    print(f"Staff zones: {len(zones)}")
    
    # Examine each staff
    for i, staff in enumerate(staffs):
        print(f"\nStaff {i}:")
        print(f"  Lines: {len(staff.lines)}")
        print(f"  Unit size: {staff.unit_size:.2f} pixels")
        print(f"  Track: {staff.track}")
        print(f"  Group: {staff.group}")
        print(f"  Center: ({staff.x_center:.1f}, {staff.y_center:.1f})")
        print(f"  Bounds: x=[{staff.x_left}, {staff.x_right}], y=[{staff.y_upper}, {staff.y_lower}]")
        print(f"  Slope: {staff.slope:.6f}")
        print(f"  Complete: {not staff.incomplete}")
        
except Exception as e:
    print(f"Staff extraction failed: {e}")

Staff Analysis and Validation

from oemer.staffline_extraction import extract, Staff, Line
import numpy as np

def analyze_staff_quality(staffs: List[Staff]) -> dict:
    """Analyze the quality and consistency of detected staffs."""
    
    analysis = {
        'total_staffs': len(staffs),
        'complete_staffs': 0,
        'incomplete_staffs': 0,
        'unit_sizes': [],
        'slopes': [],
        'tracks': set(),
        'groups': set()
    }
    
    for staff in staffs:
        if staff.incomplete:
            analysis['incomplete_staffs'] += 1
            print(f"Warning: Incomplete staff with {len(staff.lines)} lines")
        else:
            analysis['complete_staffs'] += 1
        
        analysis['unit_sizes'].append(staff.unit_size)
        analysis['slopes'].append(staff.slope)
        analysis['tracks'].add(staff.track)
        analysis['groups'].add(staff.group)
    
    # Calculate statistics
    if analysis['unit_sizes']:
        analysis['avg_unit_size'] = np.mean(analysis['unit_sizes'])
        analysis['unit_size_std'] = np.std(analysis['unit_sizes'])
        analysis['avg_slope'] = np.mean(analysis['slopes'])
        analysis['slope_std'] = np.std(analysis['slopes'])
    
    return analysis

# Run staff detection and analysis
staffs, zones = extract()
analysis = analyze_staff_quality(staffs)

print("Staff Analysis Results:")
print(f"Total staffs: {analysis['total_staffs']}")
print(f"Complete staffs: {analysis['complete_staffs']}")
print(f"Incomplete staffs: {analysis['incomplete_staffs']}")
print(f"Average unit size: {analysis.get('avg_unit_size', 0):.2f} ± {analysis.get('unit_size_std', 0):.2f}")
print(f"Average slope: {analysis.get('avg_slope', 0):.6f} ± {analysis.get('slope_std', 0):.6f}")
print(f"Tracks detected: {sorted(analysis['tracks'])}")
print(f"Groups detected: {sorted(analysis['groups'])}")

Manual Staff Construction

from oemer.staffline_extraction import Staff, Line, LineLabel

def create_staff_manually(line_positions: List[List[Tuple[int, int]]]) -> Staff:
    """Create a staff manually from line coordinate data."""
    
    staff = Staff()
    
    for i, points in enumerate(line_positions):
        if i >= 5:  # Maximum 5 lines per staff
            break
            
        line = Line()
        for y, x in points:
            line.add_point(y, x)
        
        staff.add_line(line)
    
    return staff

# Example: Create a staff with known line positions
line_data = [
    [(100, 50), (100, 100), (100, 150), (100, 200)],  # Bottom line
    [(110, 50), (110, 100), (110, 150), (110, 200)],  # Second line
    [(120, 50), (120, 100), (120, 150), (120, 200)],  # Middle line
    [(130, 50), (130, 100), (130, 150), (130, 200)],  # Fourth line
    [(140, 50), (140, 100), (140, 150), (140, 200)]   # Top line
]

custom_staff = create_staff_manually(line_data)
print(f"Custom staff unit size: {custom_staff.unit_size}")
print(f"Custom staff center: ({custom_staff.x_center}, {custom_staff.y_center})")

Multi-Staff System Processing

from oemer.staffline_extraction import extract
import matplotlib.pyplot as plt

def visualize_staff_system(staffs: List[Staff], zones: List[range]) -> None:
    """Visualize detected staff system with tracks and groups."""
    
    fig, ax = plt.subplots(figsize=(15, 10))
    
    # Color map for different groups
    colors = ['red', 'blue', 'green', 'orange', 'purple', 'brown']
    
    for staff in staffs:
        color = colors[staff.group % len(colors)]
        
        # Draw staff lines
        for line in staff.lines:
            x_coords = [p[1] for p in line.points] if hasattr(line, 'points') else [staff.x_left, staff.x_right]
            y_coords = [p[0] for p in line.points] if hasattr(line, 'points') else [line.y_center, line.y_center]
            ax.plot(x_coords, y_coords, color=color, linewidth=2, alpha=0.7)
        
        # Add staff information
        ax.text(staff.x_left - 50, staff.y_center, 
                f'T{staff.track}G{staff.group}', 
                fontsize=10, color=color, weight='bold')
    
    # Draw zone boundaries
    for i, zone in enumerate(zones):
        ax.axhspan(zone.start, zone.stop, alpha=0.1, color='gray')
        ax.text(10, (zone.start + zone.stop) / 2, f'Zone {i}', 
                rotation=90, fontsize=8)
    
    ax.set_title('Detected Staff System')
    ax.set_xlabel('X Position')
    ax.set_ylabel('Y Position')
    ax.invert_yaxis()  # Invert Y-axis to match image coordinates
    ax.grid(True, alpha=0.3)
    
    plt.tight_layout()
    plt.savefig('staff_system_visualization.png', dpi=150)
    plt.show()

# Detect and visualize staff system
staffs, zones = extract()
visualize_staff_system(staffs, zones)

Staff Detection Algorithm

Multi-Stage Processing

Image Splitting: Divide image into horizontal sections for robust detection
Peak Detection: Find horizontal accumulation peaks corresponding to staff lines
Line Grouping: Group individual lines into complete 5-line staffs
Validation: Check consistency of unit sizes and alignment
Track Assignment: Assign track numbers for multi-staff systems
Group Assignment: Group related staffs (e.g., piano grand staff)

Key Measurements

Unit Size: Average distance between adjacent staff lines
Slope: Average slope across all lines in a staff
Alignment: Horizontal consistency of staff line positions
Spacing: Vertical consistency of line spacing

Quality Metrics

Staff lines should be approximately parallel
Unit size should be consistent within ±10% across all staffs
Complete staffs should have exactly 5 lines
Track and group assignments should follow musical conventions

The staff detection system provides the foundational coordinate system for all subsequent musical element recognition, making its accuracy critical for the overall OMR pipeline performance.

Install with Tessl CLI