0
# Core Reading and Writing
1
2
Low-level CSV reader and writer classes that provide drop-in compatibility with Python's csv module while supporting CleverCSV's enhanced dialect handling. These classes form the foundation of CleverCSV's CSV processing capabilities.
3
4
## Capabilities
5
6
### Reader Class
7
8
CSV reader class that parses CSV data row by row, supporting all CleverCSV dialect features and providing an iterator interface.
9
10
```python { .api }
11
class reader:
12
"""
13
CSV reader with enhanced dialect support.
14
Drop-in replacement for csv.reader with better dialect handling.
15
"""
16
17
def __init__(
18
self,
19
csvfile: Iterable[str],
20
dialect: Union[str, SimpleDialect, csv.Dialect] = 'excel',
21
**fmtparams
22
):
23
"""
24
Initialize CSV reader.
25
26
Parameters:
27
- csvfile: File-like object or iterable of strings
28
- dialect: Dialect specification ('excel', SimpleDialect, or csv.Dialect)
29
- **fmtparams: Format parameters (delimiter, quotechar, escapechar, strict)
30
"""
31
32
def __iter__(self) -> Iterator[List[str]]:
33
"""Return iterator over rows."""
34
35
def __next__(self) -> List[str]:
36
"""
37
Return next row as list of strings.
38
39
Returns:
40
List of field values as strings
41
42
Raises:
43
StopIteration: When no more rows available
44
Error: When parsing error occurs
45
"""
46
47
@property
48
def dialect(self) -> csv.Dialect:
49
"""Current dialect used for reading."""
50
51
@property
52
def line_num(self) -> int:
53
"""Current line number being processed."""
54
```
55
56
#### Usage Examples
57
58
```python
59
import clevercsv
60
61
# Basic usage with file
62
with open('data.csv', 'r', newline='') as f:
63
reader = clevercsv.reader(f)
64
for row in reader:
65
print(row)
66
67
# With automatic dialect detection
68
with open('data.csv', 'r', newline='') as f:
69
sample = f.read()
70
dialect = clevercsv.Detector().detect(sample)
71
f.seek(0)
72
reader = clevercsv.reader(f, dialect=dialect)
73
rows = list(reader)
74
75
# With custom dialect
76
dialect = clevercsv.SimpleDialect(',', '"', '\\')
77
with open('data.csv', 'r', newline='') as f:
78
reader = clevercsv.reader(f, dialect=dialect)
79
header = next(reader) # First row
80
data_rows = list(reader) # Remaining rows
81
82
# With format parameters
83
with open('data.csv', 'r', newline='') as f:
84
reader = clevercsv.reader(f, delimiter=';', quotechar="'")
85
for i, row in enumerate(reader):
86
print(f"Row {reader.line_num}: {row}")
87
if i >= 10: # First 10 rows only
88
break
89
```
90
91
### Writer Class
92
93
CSV writer class that formats and writes CSV data, supporting all CleverCSV dialect features and maintaining compatibility with csv.writer interface.
94
95
```python { .api }
96
class writer:
97
"""
98
CSV writer with enhanced dialect support.
99
Drop-in replacement for csv.writer with better dialect handling.
100
"""
101
102
def __init__(
103
self,
104
csvfile: SupportsWrite[str],
105
dialect: Union[str, SimpleDialect, csv.Dialect] = 'excel',
106
**fmtparams
107
):
108
"""
109
Initialize CSV writer.
110
111
Parameters:
112
- csvfile: File-like object that supports writing
113
- dialect: Dialect specification ('excel', SimpleDialect, or csv.Dialect)
114
- **fmtparams: Format parameters (delimiter, quotechar, escapechar, etc.)
115
"""
116
117
def writerow(self, row: Iterable[Any]) -> Any:
118
"""
119
Write a single row to the CSV file.
120
121
Parameters:
122
- row: Iterable of values to write
123
124
Returns:
125
Return value from underlying csv.writer.writerow()
126
127
Raises:
128
Error: When writing fails
129
"""
130
131
def writerows(self, rows: Iterable[Iterable[Any]]) -> Any:
132
"""
133
Write multiple rows to the CSV file.
134
135
Parameters:
136
- rows: Iterable of rows, each row is an iterable of values
137
138
Returns:
139
Return value from underlying csv.writer.writerows()
140
141
Raises:
142
Error: When writing fails
143
"""
144
145
@property
146
def dialect(self) -> Type[csv.Dialect]:
147
"""Current dialect used for writing."""
148
```
149
150
#### Usage Examples
151
152
```python
153
import clevercsv
154
155
# Basic writing
156
data = [
157
['Name', 'Age', 'City'],
158
['Alice', '30', 'New York'],
159
['Bob', '25', 'San Francisco']
160
]
161
162
with open('output.csv', 'w', newline='') as f:
163
writer = clevercsv.writer(f)
164
writer.writerows(data)
165
166
# Write with specific dialect
167
dialect = clevercsv.SimpleDialect(';', '"', '')
168
with open('output.csv', 'w', newline='') as f:
169
writer = clevercsv.writer(f, dialect=dialect)
170
writer.writerow(['col1', 'col2', 'col3'])
171
writer.writerow(['value1', 'value2', 'value3'])
172
173
# Write with format parameters
174
with open('output.csv', 'w', newline='') as f:
175
writer = clevercsv.writer(f, delimiter='|', quotechar="'")
176
for row in data:
177
writer.writerow(row)
178
179
# Write using detected dialect from input file
180
input_dialect = clevercsv.detect_dialect('input.csv')
181
with open('output.csv', 'w', newline='') as f:
182
writer = clevercsv.writer(f, dialect=input_dialect)
183
writer.writerows(processed_data)
184
```
185
186
## Advanced Usage Patterns
187
188
### Stream Processing
189
190
Process large CSV files without loading everything into memory:
191
192
```python
193
import clevercsv
194
195
def process_large_csv(input_file, output_file, transform_func):
196
"""Process CSV file row by row with transformation."""
197
198
# Detect dialect from input
199
dialect = clevercsv.detect_dialect(input_file)
200
201
with open(input_file, 'r', newline='') as infile, \
202
open(output_file, 'w', newline='') as outfile:
203
204
reader = clevercsv.reader(infile, dialect=dialect)
205
writer = clevercsv.writer(outfile, dialect='excel') # Standardize output
206
207
# Process header
208
header = next(reader)
209
writer.writerow(transform_func(header))
210
211
# Process data rows
212
for row in reader:
213
transformed_row = transform_func(row)
214
writer.writerow(transformed_row)
215
216
# Usage
217
process_large_csv('input.csv', 'output.csv', lambda row: [cell.upper() for cell in row])
218
```
219
220
### Dialect Preservation
221
222
Preserve input dialect when processing files:
223
224
```python
225
import clevercsv
226
227
def preserve_dialect_processing(filename):
228
"""Process CSV while preserving original dialect."""
229
230
# Detect original dialect
231
original_dialect = clevercsv.detect_dialect(filename)
232
233
# Read data
234
with open(filename, 'r', newline='') as f:
235
reader = clevercsv.reader(f, dialect=original_dialect)
236
data = list(reader)
237
238
# Process data
239
processed_data = process_data(data)
240
241
# Write back with same dialect
242
with open(filename, 'w', newline='') as f:
243
writer = clevercsv.writer(f, dialect=original_dialect)
244
writer.writerows(processed_data)
245
```
246
247
### Error Handling
248
249
Robust error handling for CSV processing:
250
251
```python
252
import clevercsv
253
254
def robust_csv_processing(filename):
255
"""CSV processing with comprehensive error handling."""
256
257
try:
258
# Attempt dialect detection
259
dialect = clevercsv.detect_dialect(filename)
260
if dialect is None:
261
raise ValueError("Could not detect CSV dialect")
262
263
with open(filename, 'r', newline='') as f:
264
reader = clevercsv.reader(f, dialect=dialect)
265
266
try:
267
for line_num, row in enumerate(reader, 1):
268
try:
269
# Process row
270
process_row(row)
271
except Exception as e:
272
print(f"Error processing row {line_num}: {e}")
273
continue
274
275
except clevercsv.Error as e:
276
print(f"CSV parsing error at line {reader.line_num}: {e}")
277
278
except FileNotFoundError:
279
print(f"File not found: {filename}")
280
except Exception as e:
281
print(f"Unexpected error: {e}")
282
```
283
284
## Performance Considerations
285
286
### Memory Efficiency
287
288
```python
289
# Memory-efficient: process row by row
290
with open('large_file.csv', 'r', newline='') as f:
291
reader = clevercsv.reader(f)
292
for row in reader:
293
process_row(row) # Process immediately, don't store
294
295
# Memory-intensive: load all at once
296
with open('large_file.csv', 'r', newline='') as f:
297
reader = clevercsv.reader(f)
298
all_rows = list(reader) # Loads entire file into memory
299
```
300
301
### Speed Optimization
302
303
```python
304
# Faster: Use SimpleDialect directly if known
305
known_dialect = clevercsv.SimpleDialect(',', '"', '')
306
reader = clevercsv.reader(file, dialect=known_dialect)
307
308
# Slower: String dialect requires lookup
309
reader = clevercsv.reader(file, dialect='excel')
310
```
311
312
## Integration with csv Module
313
314
CleverCSV readers and writers are designed as drop-in replacements:
315
316
```python
317
# Standard csv module
318
import csv
319
with open('data.csv', 'r', newline='') as f:
320
reader = csv.reader(f)
321
data = list(reader)
322
323
# CleverCSV replacement
324
import clevercsv
325
with open('data.csv', 'r', newline='') as f:
326
reader = clevercsv.reader(f) # Enhanced dialect handling
327
data = list(reader)
328
329
# Mixed usage
330
import clevercsv
331
import csv
332
333
# Detect with CleverCSV, read with standard csv
334
dialect = clevercsv.detect_dialect('data.csv')
335
with open('data.csv', 'r', newline='') as f:
336
reader = csv.reader(f, dialect=dialect.to_csv_dialect())
337
data = list(reader)
338
```
339
340
## Format Parameters
341
342
Both reader and writer support the same format parameters as the csv module:
343
344
- `delimiter`: Field separator character
345
- `quotechar`: Character used to quote fields containing special characters
346
- `escapechar`: Character used to escape delimiter/quote characters
347
- `strict`: Whether to raise exceptions on bad CSV input
348
- `skipinitialspace`: Whether to ignore whitespace after delimiter
349
- `quoting`: Quoting behavior (QUOTE_MINIMAL, QUOTE_ALL, etc.)
350
351
```python
352
# Example with multiple format parameters
353
reader = clevercsv.reader(
354
file,
355
delimiter=';',
356
quotechar="'",
357
escapechar='\\',
358
strict=True
359
)
360
```