0
# Notehead Extraction
1

2
Extraction and analysis of musical noteheads from neural network predictions. This module identifies individual noteheads, determines their properties, and associates them with musical context.
3

4
## Capabilities
5

6
### Primary Notehead Extraction
7

8
Extract noteheads from neural network predictions and create structured representations.
9

10
```python { .api }
11
def extract() -> List[NoteHead]:
12
    """
13
    Extract noteheads from neural network predictions.
14
    
15
    Processes notehead predictions from the layer system, identifies
16
    individual notehead regions, and creates NoteHead instances with
17
    properties like position, pitch, and rhythm type.
18
    
19
    Returns:
20
    List[NoteHead]: List of detected and analyzed noteheads
21
    
22
    Raises:
23
    KeyError: If required prediction layers are not available
24
    """
25
```
26

27
### Notehead Classification
28

29
Functions for refining notehead detection and classification.
30

31
```python { .api }
32
def get_predict_class(region: ndarray) -> NoteType:
33
    """
34
    Predict the note type (rhythm) from a notehead region.
35
    
36
    Uses trained sklearn models to classify noteheads as whole, half,
37
    quarter, eighth, etc. based on visual characteristics.
38
    
39
    Parameters:
40
    - region (ndarray): Image region containing the notehead
41
    
42
    Returns:
43
    NoteType: Predicted note type/rhythm classification
44
    """
45

46
def morph_notehead(note: NoteHead, size: Tuple[int, int] = (11, 8)) -> ndarray:
47
    """
48
    Apply morphological operations to clean notehead region.
49
    
50
    Parameters:
51
    - note (NoteHead): The notehead to process
52
    - size (Tuple[int, int]): Kernel size for morphological operations
53
    
54
    Returns:
55
    ndarray: Cleaned notehead region
56
    """
57
```
58

59
### Notehead Analysis
60

61
Functions for analyzing notehead properties and context.
62

63
```python { .api }
64
def get_symbol_regression(note: NoteHead, staff: Staff) -> int:
65
    """
66
    Determine the pitch position of a notehead relative to staff lines.
67
    
68
    Calculates the position of the notehead center relative to the
69
    five staff lines to determine pitch.
70
    
71
    Parameters:
72
    - note (NoteHead): The notehead to analyze
73
    - staff (Staff): The staff containing this notehead
74
    
75
    Returns:
76
    int: Staff line position (negative for below bottom line,
77
         0-4 for on staff lines, positive for above top line)
78
    """
79

80
def check_stem_up_down(note: NoteHead) -> Optional[bool]:
81
    """
82
    Determine stem direction for a notehead.
83
    
84
    Analyzes the surrounding region to detect stem presence
85
    and direction (up or down).
86
    
87
    Parameters:
88
    - note (NoteHead): The notehead to analyze
89
    
90
    Returns:
91
    Optional[bool]: True for stem up, False for stem down, 
92
                   None if no stem detected
93
    """
94

95
def check_dots(note: NoteHead) -> bool:
96
    """
97
    Check if a notehead has augmentation dots.
98
    
99
    Searches the region to the right of the notehead for
100
    augmentation dots that extend the note's duration.
101
    
102
    Parameters:
103
    - note (NoteHead): The notehead to check
104
    
105
    Returns:
106
    bool: True if dots are detected, False otherwise
107
    """
108
```
109

110
## NoteHead Class
111

112
Complete representation of a musical notehead with all associated properties.
113

114
```python { .api }
115
class NoteHead:
116
    """
117
    Represents a musical notehead with rhythm, pitch, and context information.
118
    
119
    Attributes:
120
    - points (List[Tuple[int, int]]): Pixel coordinates belonging to this notehead
121
    - pitch (Optional[int]): MIDI pitch number (None if not determined)
122
    - has_dot (bool): Whether this note has augmentation dots
123
    - bbox (BBox): Bounding box coordinates (x1, y1, x2, y2)
124
    - stem_up (Optional[bool]): Stem direction (True=up, False=down, None=no stem)
125
    - stem_right (Optional[bool]): Whether stem is on right side of notehead
126
    - track (Optional[int]): Track number for multi-staff systems
127
    - group (Optional[int]): Group number for staff groupings
128
    - staff_line_pos (int): Position relative to staff lines
129
    - invalid (bool): Whether this detection is likely a false positive
130
    - id (Optional[int]): Unique identifier within the document
131
    - note_group_id (Optional[int]): ID of the note group this belongs to
132
    - sfn (Optional[Any]): Associated sharp/flat/natural accidental
133
    - label (NoteType): Rhythm classification (whole, half, quarter, etc.)
134
    """
135
    
136
    def add_point(self, x: int, y: int) -> None:
137
        """
138
        Add a pixel coordinate to this notehead.
139
        
140
        Parameters:
141
        - x (int): X coordinate
142
        - y (int): Y coordinate
143
        """
144
    
145
    def force_set_label(self, label: NoteType) -> None:
146
        """
147
        Force set the note type label, overriding any existing label.
148
        
149
        Parameters:
150
        - label (NoteType): The note type to assign
151
        
152
        Raises:
153
        AssertionError: If label is not a NoteType instance
154
        """
155
    
156
    @property
157
    def label(self) -> Optional[NoteType]:
158
        """
159
        Get the note type label.
160
        
161
        Returns None if the notehead is marked as invalid.
162
        
163
        Returns:
164
        Optional[NoteType]: The note type or None if invalid
165
        """
166
    
167
    @label.setter
168
    def label(self, label: NoteType) -> None:
169
        """
170
        Set the note type label.
171
        
172
        Will not overwrite existing labels unless using force_set_label.
173
        
174
        Parameters:
175
        - label (NoteType): The note type to assign
176
        """
177
```
178

179
## NoteType Enumeration
180

181
Classification of note types by rhythm duration.
182

183
```python { .api }
184
class NoteType(enum.Enum):
185
    """
186
    Enumeration of note types by rhythmic duration.
187
    """
188
    WHOLE = 0           # Whole note (4 beats)
189
    HALF = 1            # Half note (2 beats)  
190
    QUARTER = 2         # Quarter note (1 beat)
191
    EIGHTH = 3          # Eighth note (1/2 beat)
192
    SIXTEENTH = 4       # Sixteenth note (1/4 beat)
193
    THIRTY_SECOND = 5   # Thirty-second note (1/8 beat)
194
    SIXTY_FOURTH = 6    # Sixty-fourth note (1/16 beat)
195
    TRIPLET = 7         # Triplet note (special timing)
196
    OTHERS = 8          # Other/unclassified note types
197
    HALF_OR_WHOLE = 9   # Intermediate classification state
198
```
199

200
## Usage Examples
201

202
### Basic Notehead Extraction
203

204
```python
205
from oemer.notehead_extraction import extract
206
from oemer.layers import register_layer, get_layer
207
import numpy as np
208

209
# Assume neural network predictions are already in layer system
210
# (This would normally be done by the inference module)
211

212
# Extract noteheads
213
try:
214
    noteheads = extract()
215
    print(f"Extracted {len(noteheads)} noteheads")
216
    
217
    # Analyze each notehead
218
    for i, note in enumerate(noteheads):
219
        print(f"\nNotehead {i}:")
220
        print(f"  Position: {note.bbox}")
221
        print(f"  Pitch: {note.pitch}")
222
        print(f"  Type: {note.label}")
223
        print(f"  Has dot: {note.has_dot}")
224
        print(f"  Stem up: {note.stem_up}")
225
        print(f"  Track: {note.track}")
226
        print(f"  Valid: {not note.invalid}")
227
        
228
except KeyError as e:
229
    print(f"Missing required layer: {e}")
230
```
231

232
### Notehead Analysis and Filtering
233

234
```python
235
from oemer.notehead_extraction import extract, get_symbol_regression, check_dots
236
from oemer.staffline_extraction import extract as staff_extract
237

238
# Extract staffs and noteheads
239
staffs, _ = staff_extract()
240
noteheads = extract()
241

242
# Analyze noteheads in context of staffs
243
valid_notes = []
244
for note in noteheads:
245
    if note.invalid:
246
        continue
247
        
248
    # Find the closest staff
249
    closest_staff = None
250
    min_distance = float('inf')
251
    
252
    for staff in staffs:
253
        distance = abs(note.bbox[1] - staff.y_center)
254
        if distance < min_distance:
255
            min_distance = distance
256
            closest_staff = staff
257
    
258
    if closest_staff:
259
        # Get pitch position relative to staff
260
        staff_pos = get_symbol_regression(note, closest_staff)
261
        note.staff_line_pos = staff_pos
262
        
263
        # Check for augmentation dots
264
        has_dots = check_dots(note)
265
        note.has_dot = has_dots
266
        
267
        valid_notes.append(note)
268

269
print(f"Valid noteheads: {len(valid_notes)}")
270

271
# Group by note type
272
from collections import defaultdict
273
by_type = defaultdict(list)
274
for note in valid_notes:
275
    by_type[note.label].append(note)
276

277
for note_type, notes in by_type.items():
278
    print(f"{note_type.name}: {len(notes)} notes")
279
```
280

281
### Custom Notehead Classification
282

283
```python
284
from oemer.notehead_extraction import extract, get_predict_class, morph_notehead
285
import cv2
286

287
# Extract noteheads
288
noteheads = extract()
289

290
# Reclassify noteheads with custom processing
291
for note in noteheads:
292
    if note.invalid:
293
        continue
294
        
295
    # Get the image region for this notehead
296
    image = get_layer("original_image")
297
    x1, y1, x2, y2 = note.bbox
298
    region = image[y1:y2, x1:x2]
299
    
300
    # Apply morphological cleaning
301
    cleaned_region = morph_notehead(note, size=(11, 8))
302
    
303
    # Reclassify the note type
304
    predicted_type = get_predict_class(cleaned_region)
305
    
306
    # Update the note if classification differs significantly
307
    if note.label != predicted_type:
308
        print(f"Note {note.id}: {note.label} -> {predicted_type}")
309
        note.force_set_label(predicted_type)
310
```
311

312
## Processing Pipeline Integration
313

314
The notehead extraction module integrates with the main OMR pipeline:
315

316
1. **Input**: Neural network predictions from `notehead_pred` layer
317
2. **Staff Context**: Uses staff information from `staffs` layer 
318
3. **Image Data**: Accesses original image from `original_image` layer
319
4. **Output**: Registers extracted noteheads in `notes` layer
320
5. **ID Mapping**: Creates pixel-to-note mapping in `note_id` layer
321

322
The extracted noteheads are then used by:
323
- Note grouping module to form chords and beamed groups
324
- Rhythm extraction module to determine final timing
325
- Symbol extraction module to associate accidentals
326
- MusicXML builder to generate structured output
327

328
This modular design allows for independent testing and refinement of each processing stage while maintaining integration with the overall pipeline.