0
# Staffline Detection and Analysis
1

2
Detection and analysis of musical staff lines, which form the foundation for all subsequent processing steps. Staff lines provide the coordinate system for pitch interpretation and serve as reference points for symbol positioning.
3

4
## Capabilities
5

6
### Primary Staff Extraction
7

8
Extract staff lines and organize them into complete staff structures.
9

10
```python { .api }
11
def extract(splits: int = 8, line_threshold: float = 0.8, horizontal_diff_th: float = 0.1, unit_size_diff_th: float = 0.1, barline_min_degree: int = 75) -> Tuple[ndarray, ndarray]:
12
    """
13
    Main staff extraction function that detects and analyzes staff lines.
14
    
15
    Parameters:
16
    - splits (int): Number of horizontal splits for processing (default: 8)
17
    - line_threshold (float): Threshold for staff line detection (default: 0.8)
18
    - horizontal_diff_th (float): Threshold for horizontal alignment validation (default: 0.1)
19
    - unit_size_diff_th (float): Threshold for unit size consistency (default: 0.1)
20
    - barline_min_degree (int): Minimum angle in degrees for barline detection (default: 75)
21
    
22
    Returns:
23
    Tuple containing:
24
    - staffs (ndarray): Array of Staff instances
25
    - zones (ndarray): Array of zone boundaries for each staff group
26
    
27
    Raises:
28
    StafflineException: If staff detection fails or results are inconsistent
29
    """
30

31
def extract_part(pred: ndarray, x_offset: int, line_threshold: float = 0.8) -> List[Staff]:
32
    """
33
    Extract staff structures from a specific image region.
34
    
35
    Parameters:
36
    - pred (ndarray): Binary prediction array for the image region
37
    - x_offset (int): Horizontal offset of this region in the full image
38
    - line_threshold (float): Detection threshold for staff lines
39
    
40
    Returns:
41
    List[Staff]: List of detected Staff instances in this region
42
    """
43

44
def extract_line(pred: ndarray, x_offset: int, line_threshold: float = 0.8) -> Tuple[ndarray, ndarray]:
45
    """
46
    Extract individual staff lines from prediction data.
47
    
48
    Parameters:
49
    - pred (ndarray): Binary prediction array
50
    - x_offset (int): Horizontal offset in the full image
51
    - line_threshold (float): Detection threshold
52
    
53
    Returns:
54
    Tuple containing line positions and metadata
55
    """
56
```
57

58
### Staff Line Representation
59

60
Individual staff line with position and geometric properties.
61

62
```python { .api }
63
class Line:
64
    """
65
    Represents a single staff line with geometric properties.
66
    
67
    Attributes:
68
    - points (List[Tuple[int, int]]): List of (x, y) coordinates on the line
69
    """
70
    
71
    def add_point(self, y: int, x: int) -> None:
72
        """
73
        Add a coordinate point to this staff line.
74
        
75
        Parameters:
76
        - y (int): Y coordinate of the point
77
        - x (int): X coordinate of the point
78
        """
79
    
80
    @property
81
    def y_center(self) -> float:
82
        """Get the average Y coordinate of all points on this line."""
83
    
84
    @property
85
    def y_upper(self) -> int:
86
        """Get the minimum Y coordinate (top) of this line."""
87
    
88
    @property
89
    def y_lower(self) -> int:
90
        """Get the maximum Y coordinate (bottom) of this line."""
91
    
92
    @property
93
    def x_center(self) -> float:
94
        """Get the average X coordinate of all points on this line."""
95
    
96
    @property
97
    def x_left(self) -> int:
98
        """Get the minimum X coordinate (left) of this line."""
99
    
100
    @property
101
    def x_right(self) -> int:
102
        """Get the maximum X coordinate (right) of this line."""
103
    
104
    @property
105
    def slope(self) -> float:
106
        """Get the estimated slope of this line."""
107
```
108

109
### Complete Staff Representation
110

111
A complete musical staff consisting of five staff lines.
112

113
```python { .api }
114
class Staff:
115
    """
116
    Represents a complete musical staff with five staff lines.
117
    
118
    Attributes:
119
    - lines (List[Line]): List of the five staff lines (bottom to top)
120
    - track (int): Track number for multi-staff systems
121
    - group (int): Group number for staff groupings (e.g., piano grand staff)
122
    - is_interp (bool): Whether this staff was interpolated from partial data
123
    """
124
    
125
    def add_line(self, line: Line) -> None:
126
        """
127
        Add a staff line to this staff.
128
        
129
        Parameters:
130
        - line (Line): The staff line to add
131
        
132
        Raises:
133
        ValueError: If trying to add more than 5 lines
134
        """
135
    
136
    def duplicate(self, x_offset: int = 0, y_offset: int = 0) -> 'Staff':
137
        """
138
        Create a duplicate of this staff with optional offset.
139
        
140
        Parameters:
141
        - x_offset (int): Horizontal offset for the duplicate (default: 0)
142
        - y_offset (int): Vertical offset for the duplicate (default: 0)
143
        
144
        Returns:
145
        Staff: New Staff instance with offset applied
146
        """
147
    
148
    @property
149
    def y_center(self) -> float:
150
        """Get the vertical center of this staff (middle of the 5 lines)."""
151
    
152
    @property
153
    def y_upper(self) -> int:
154
        """Get the top boundary of this staff."""
155
    
156
    @property
157
    def y_lower(self) -> int:
158
        """Get the bottom boundary of this staff."""
159
    
160
    @property
161
    def x_center(self) -> float:
162
        """Get the horizontal center of this staff."""
163
    
164
    @property
165
    def x_left(self) -> int:
166
        """Get the left boundary of this staff."""
167
    
168
    @property
169
    def x_right(self) -> int:
170
        """Get the right boundary of this staff."""
171
    
172
    @property
173
    def unit_size(self) -> float:
174
        """
175
        Get the unit size (average distance between staff lines).
176
        
177
        This is the fundamental measurement unit for all musical elements.
178
        All other measurements (note sizes, stem lengths, etc.) are 
179
        proportional to this unit size.
180
        """
181
    
182
    @property
183
    def incomplete(self) -> bool:
184
        """Check if this staff has fewer than 5 lines."""
185
    
186
    @property
187
    def slope(self) -> float:
188
        """Get the average slope of all lines in this staff."""
189
```
190

191
### Staff Line Position Enumeration
192

193
Enumeration for staff line positions within a staff.
194

195
```python { .api }
196
class LineLabel(enum.Enum):
197
    """
198
    Enumeration for staff line positions.
199
    
200
    Values represent the five staff lines from bottom to top.
201
    """
202
    FIRST = 0    # Bottom staff line
203
    SECOND = 1   # Second staff line from bottom
204
    THIRD = 2    # Middle staff line
205
    FOURTH = 3   # Fourth staff line from bottom
206
    FIFTH = 4    # Top staff line
207
```
208

209
## Usage Examples
210

211
### Basic Staff Detection
212

213
```python
214
from oemer.staffline_extraction import extract
215
from oemer.layers import register_layer, get_layer
216
import numpy as np
217
import cv2
218

219
# Load and prepare image
220
image = cv2.imread("sheet_music.jpg")
221
register_layer("original_image", image)
222

223
# Simulate staff predictions (in real usage, this comes from neural network)
224
h, w = image.shape[:2]
225
staff_pred = np.random.randint(0, 2, (h, w), dtype=np.uint8)
226
register_layer("staff_pred", staff_pred)
227

228
# Extract staff lines
229
try:
230
    staffs, zones = extract(
231
        splits=8,           # Process in 8 horizontal sections
232
        line_threshold=0.8, # High confidence threshold
233
        horizontal_diff_th=0.1,  # Strict alignment requirement
234
        unit_size_diff_th=0.1    # Consistent spacing requirement
235
    )
236
    
237
    print(f"Detected {len(staffs)} staff structures")
238
    print(f"Staff zones: {len(zones)}")
239
    
240
    # Examine each staff
241
    for i, staff in enumerate(staffs):
242
        print(f"\nStaff {i}:")
243
        print(f"  Lines: {len(staff.lines)}")
244
        print(f"  Unit size: {staff.unit_size:.2f} pixels")
245
        print(f"  Track: {staff.track}")
246
        print(f"  Group: {staff.group}")
247
        print(f"  Center: ({staff.x_center:.1f}, {staff.y_center:.1f})")
248
        print(f"  Bounds: x=[{staff.x_left}, {staff.x_right}], y=[{staff.y_upper}, {staff.y_lower}]")
249
        print(f"  Slope: {staff.slope:.6f}")
250
        print(f"  Complete: {not staff.incomplete}")
251
        
252
except Exception as e:
253
    print(f"Staff extraction failed: {e}")
254
```
255

256
### Staff Analysis and Validation
257

258
```python
259
from oemer.staffline_extraction import extract, Staff, Line
260
import numpy as np
261

262
def analyze_staff_quality(staffs: List[Staff]) -> dict:
263
    """Analyze the quality and consistency of detected staffs."""
264
    
265
    analysis = {
266
        'total_staffs': len(staffs),
267
        'complete_staffs': 0,
268
        'incomplete_staffs': 0,
269
        'unit_sizes': [],
270
        'slopes': [],
271
        'tracks': set(),
272
        'groups': set()
273
    }
274
    
275
    for staff in staffs:
276
        if staff.incomplete:
277
            analysis['incomplete_staffs'] += 1
278
            print(f"Warning: Incomplete staff with {len(staff.lines)} lines")
279
        else:
280
            analysis['complete_staffs'] += 1
281
        
282
        analysis['unit_sizes'].append(staff.unit_size)
283
        analysis['slopes'].append(staff.slope)
284
        analysis['tracks'].add(staff.track)
285
        analysis['groups'].add(staff.group)
286
    
287
    # Calculate statistics
288
    if analysis['unit_sizes']:
289
        analysis['avg_unit_size'] = np.mean(analysis['unit_sizes'])
290
        analysis['unit_size_std'] = np.std(analysis['unit_sizes'])
291
        analysis['avg_slope'] = np.mean(analysis['slopes'])
292
        analysis['slope_std'] = np.std(analysis['slopes'])
293
    
294
    return analysis
295

296
# Run staff detection and analysis
297
staffs, zones = extract()
298
analysis = analyze_staff_quality(staffs)
299

300
print("Staff Analysis Results:")
301
print(f"Total staffs: {analysis['total_staffs']}")
302
print(f"Complete staffs: {analysis['complete_staffs']}")
303
print(f"Incomplete staffs: {analysis['incomplete_staffs']}")
304
print(f"Average unit size: {analysis.get('avg_unit_size', 0):.2f} ± {analysis.get('unit_size_std', 0):.2f}")
305
print(f"Average slope: {analysis.get('avg_slope', 0):.6f} ± {analysis.get('slope_std', 0):.6f}")
306
print(f"Tracks detected: {sorted(analysis['tracks'])}")
307
print(f"Groups detected: {sorted(analysis['groups'])}")
308
```
309

310
### Manual Staff Construction
311

312
```python
313
from oemer.staffline_extraction import Staff, Line, LineLabel
314

315
def create_staff_manually(line_positions: List[List[Tuple[int, int]]]) -> Staff:
316
    """Create a staff manually from line coordinate data."""
317
    
318
    staff = Staff()
319
    
320
    for i, points in enumerate(line_positions):
321
        if i >= 5:  # Maximum 5 lines per staff
322
            break
323
            
324
        line = Line()
325
        for y, x in points:
326
            line.add_point(y, x)
327
        
328
        staff.add_line(line)
329
    
330
    return staff
331

332
# Example: Create a staff with known line positions
333
line_data = [
334
    [(100, 50), (100, 100), (100, 150), (100, 200)],  # Bottom line
335
    [(110, 50), (110, 100), (110, 150), (110, 200)],  # Second line
336
    [(120, 50), (120, 100), (120, 150), (120, 200)],  # Middle line
337
    [(130, 50), (130, 100), (130, 150), (130, 200)],  # Fourth line
338
    [(140, 50), (140, 100), (140, 150), (140, 200)]   # Top line
339
]
340

341
custom_staff = create_staff_manually(line_data)
342
print(f"Custom staff unit size: {custom_staff.unit_size}")
343
print(f"Custom staff center: ({custom_staff.x_center}, {custom_staff.y_center})")
344
```
345

346
### Multi-Staff System Processing
347

348
```python
349
from oemer.staffline_extraction import extract
350
import matplotlib.pyplot as plt
351

352
def visualize_staff_system(staffs: List[Staff], zones: List[range]) -> None:
353
    """Visualize detected staff system with tracks and groups."""
354
    
355
    fig, ax = plt.subplots(figsize=(15, 10))
356
    
357
    # Color map for different groups
358
    colors = ['red', 'blue', 'green', 'orange', 'purple', 'brown']
359
    
360
    for staff in staffs:
361
        color = colors[staff.group % len(colors)]
362
        
363
        # Draw staff lines
364
        for line in staff.lines:
365
            x_coords = [p[1] for p in line.points] if hasattr(line, 'points') else [staff.x_left, staff.x_right]
366
            y_coords = [p[0] for p in line.points] if hasattr(line, 'points') else [line.y_center, line.y_center]
367
            ax.plot(x_coords, y_coords, color=color, linewidth=2, alpha=0.7)
368
        
369
        # Add staff information
370
        ax.text(staff.x_left - 50, staff.y_center, 
371
                f'T{staff.track}G{staff.group}', 
372
                fontsize=10, color=color, weight='bold')
373
    
374
    # Draw zone boundaries
375
    for i, zone in enumerate(zones):
376
        ax.axhspan(zone.start, zone.stop, alpha=0.1, color='gray')
377
        ax.text(10, (zone.start + zone.stop) / 2, f'Zone {i}', 
378
                rotation=90, fontsize=8)
379
    
380
    ax.set_title('Detected Staff System')
381
    ax.set_xlabel('X Position')
382
    ax.set_ylabel('Y Position')
383
    ax.invert_yaxis()  # Invert Y-axis to match image coordinates
384
    ax.grid(True, alpha=0.3)
385
    
386
    plt.tight_layout()
387
    plt.savefig('staff_system_visualization.png', dpi=150)
388
    plt.show()
389

390
# Detect and visualize staff system
391
staffs, zones = extract()
392
visualize_staff_system(staffs, zones)
393
```
394

395
## Staff Detection Algorithm
396

397
### Multi-Stage Processing
398

399
1. **Image Splitting**: Divide image into horizontal sections for robust detection
400
2. **Peak Detection**: Find horizontal accumulation peaks corresponding to staff lines
401
3. **Line Grouping**: Group individual lines into complete 5-line staffs
402
4. **Validation**: Check consistency of unit sizes and alignment
403
5. **Track Assignment**: Assign track numbers for multi-staff systems
404
6. **Group Assignment**: Group related staffs (e.g., piano grand staff)
405

406
### Key Measurements
407

408
- **Unit Size**: Average distance between adjacent staff lines
409
- **Slope**: Average slope across all lines in a staff
410
- **Alignment**: Horizontal consistency of staff line positions
411
- **Spacing**: Vertical consistency of line spacing
412

413
### Quality Metrics
414

415
- Staff lines should be approximately parallel
416
- Unit size should be consistent within ±10% across all staffs
417
- Complete staffs should have exactly 5 lines
418
- Track and group assignments should follow musical conventions
419

420
The staff detection system provides the foundational coordinate system for all subsequent musical element recognition, making its accuracy critical for the overall OMR pipeline performance.