0
# Cloud Data Integration
1
2
Integration with major cloud data providers and services including Microsoft Planetary Computer, NASA datasets, Planet Labs, AWS Open Data, and STAC catalogs for accessing large-scale geospatial datasets.
3
4
## Capabilities
5
6
### STAC (SpatioTemporal Asset Catalog) Integration
7
8
Search and access geospatial data through STAC catalogs with comprehensive filtering and visualization capabilities.
9
10
```python { .api }
11
def stac_search(collection, bbox=None, time_range=None, **kwargs):
12
"""
13
Search STAC catalog for items matching criteria.
14
15
Args:
16
collection (str): STAC collection ID
17
bbox (list): Bounding box [minx, miny, maxx, maxy]
18
time_range (list): Time range ['start_date', 'end_date']
19
**kwargs: Additional search parameters (limit, query, etc.)
20
21
Returns:
22
list: List of matching STAC items
23
"""
24
25
def stac_assets(item, **kwargs):
26
"""
27
Get assets from STAC item.
28
29
Args:
30
item (dict): STAC item dictionary
31
**kwargs: Asset filtering options
32
33
Returns:
34
dict: Asset dictionary with URLs and metadata
35
"""
36
37
def add_stac_layer(self, url, collection=None, item=None, **kwargs):
38
"""
39
Add STAC layer to map.
40
41
Args:
42
url (str): STAC catalog, collection, or item URL
43
collection (str): Collection ID if using catalog URL
44
item (str): Item ID if using collection URL
45
**kwargs: Layer visualization options (assets, bands, rescale, etc.)
46
"""
47
48
def stac_tile(item, assets, **kwargs):
49
"""
50
Generate tiles from STAC item assets.
51
52
Args:
53
item (dict): STAC item
54
assets (list): Asset names to use for tiling
55
**kwargs: Tile generation options (rescale, colormap, etc.)
56
57
Returns:
58
str: Tile URL template
59
"""
60
```
61
62
### Microsoft Planetary Computer
63
64
Access Microsoft Planetary Computer datasets including Landsat, Sentinel, MODIS, and other Earth observation data.
65
66
```python { .api }
67
def get_pc_collections():
68
"""
69
Get list of available Planetary Computer collections.
70
71
Returns:
72
list: Collection names and descriptions
73
"""
74
75
def get_first_item(collection):
76
"""
77
Get first item from Planetary Computer collection.
78
79
Args:
80
collection (str): Collection ID
81
82
Returns:
83
dict: First STAC item in collection
84
"""
85
86
def get_bands(collection):
87
"""
88
Get available bands for Planetary Computer collection.
89
90
Args:
91
collection (str): Collection ID
92
93
Returns:
94
list: Band names and descriptions
95
"""
96
97
def get_pc_inventory(collection):
98
"""
99
Get inventory information for PC collection.
100
101
Args:
102
collection (str): Collection ID
103
104
Returns:
105
dict: Collection inventory metadata
106
"""
107
108
def pc_search(collection, bbox, time_range, **kwargs):
109
"""
110
Search Planetary Computer for data.
111
112
Args:
113
collection (str): PC collection name
114
bbox (list): Bounding box coordinates
115
time_range (list): Date range for search
116
**kwargs: Additional search parameters
117
118
Returns:
119
list: Matching STAC items
120
"""
121
```
122
123
### NASA Data Integration
124
125
Access NASA Earth observation datasets including MODIS, VIIRS, and other satellite data products.
126
127
```python { .api }
128
def nasa_data_search(dataset, bbox=None, time_range=None, **kwargs):
129
"""
130
Search NASA datasets.
131
132
Args:
133
dataset (str): NASA dataset identifier
134
bbox (list): Bounding box for spatial filtering
135
time_range (list): Time range for temporal filtering
136
**kwargs: Additional search parameters
137
138
Returns:
139
list: Matching dataset items
140
"""
141
142
def add_nasa_data(self, dataset, **kwargs):
143
"""
144
Add NASA dataset to map.
145
146
Args:
147
dataset (str): NASA dataset identifier or URL
148
**kwargs: Visualization options
149
"""
150
151
def nasa_cmr_search(short_name, **kwargs):
152
"""
153
Search NASA Common Metadata Repository.
154
155
Args:
156
short_name (str): Dataset short name
157
**kwargs: CMR search parameters (temporal, spatial, etc.)
158
159
Returns:
160
list: CMR search results
161
"""
162
```
163
164
### Planet Labs Integration
165
166
Access Planet Labs satellite imagery and data products including PlanetScope, RapidEye, and SkySat.
167
168
```python { .api }
169
def planet_tiles(item_type, **kwargs):
170
"""
171
Get Planet Labs tile URLs.
172
173
Args:
174
item_type (str): Planet item type (PSScene4Band, etc.)
175
**kwargs: Planet API parameters
176
177
Returns:
178
dict: Tile URL templates and metadata
179
"""
180
181
def planet_search(item_types, bbox, date_range, **kwargs):
182
"""
183
Search Planet Labs catalog.
184
185
Args:
186
item_types (list): Planet item types to search
187
bbox (list): Bounding box for search
188
date_range (list): Date range for search
189
**kwargs: Planet search parameters (cloud_cover, etc.)
190
191
Returns:
192
list: Matching Planet items
193
"""
194
195
def add_planet_layer(self, item_id, item_type, **kwargs):
196
"""
197
Add Planet Labs imagery layer.
198
199
Args:
200
item_id (str): Planet item ID
201
item_type (str): Planet item type
202
**kwargs: Layer visualization options
203
"""
204
```
205
206
### AWS Open Data Integration
207
208
Access geospatial datasets hosted on AWS including Landsat, Sentinel, and other public datasets.
209
210
```python { .api }
211
def s3_list_objects(bucket, prefix='', **kwargs):
212
"""
213
List objects in S3 bucket.
214
215
Args:
216
bucket (str): S3 bucket name
217
prefix (str): Object key prefix for filtering
218
**kwargs: S3 listing options (max_keys, delimiter, etc.)
219
220
Returns:
221
list: S3 object keys and metadata
222
"""
223
224
def s3_download(bucket, key, filename, **kwargs):
225
"""
226
Download file from S3.
227
228
Args:
229
bucket (str): S3 bucket name
230
key (str): S3 object key
231
filename (str): Local output filename
232
**kwargs: Download options
233
234
Returns:
235
str: Path to downloaded file
236
"""
237
238
def add_s3_layer(self, bucket, key, **kwargs):
239
"""
240
Add raster layer from S3.
241
242
Args:
243
bucket (str): S3 bucket name
244
key (str): S3 object key
245
**kwargs: Layer visualization options
246
"""
247
```
248
249
### COG (Cloud Optimized GeoTIFF) Integration
250
251
Work with Cloud Optimized GeoTIFFs for efficient web-based raster visualization and analysis.
252
253
```python { .api }
254
def cog_tile(url, **kwargs):
255
"""
256
Generate tile URL template for COG.
257
258
Args:
259
url (str): COG file URL
260
**kwargs: Tile generation options (rescale, colormap, etc.)
261
262
Returns:
263
str: Tile URL template
264
"""
265
266
def cog_info(url):
267
"""
268
Get information about COG file.
269
270
Args:
271
url (str): COG file URL
272
273
Returns:
274
dict: COG metadata (bands, CRS, bounds, etc.)
275
"""
276
277
def cog_stats(url, **kwargs):
278
"""
279
Calculate statistics for COG.
280
281
Args:
282
url (str): COG file URL
283
**kwargs: Statistics options (bands, percentiles, etc.)
284
285
Returns:
286
dict: Band statistics
287
"""
288
289
def add_cog_layer(self, url, name='Untitled', **kwargs):
290
"""
291
Add COG layer to map.
292
293
Args:
294
url (str): COG file URL
295
name (str): Layer name
296
**kwargs: Visualization options (bands, rescale, colormap, etc.)
297
"""
298
```
299
300
### Overture Maps Integration
301
302
Access Overture Maps Foundation data including buildings, transportation, places, and administrative boundaries.
303
304
```python { .api }
305
def get_overture_data(bbox, theme, **kwargs):
306
"""
307
Get Overture Maps data for bounding box.
308
309
Args:
310
bbox (list): Bounding box [minx, miny, maxx, maxy]
311
theme (str): Overture theme ('buildings', 'transportation', 'places', 'admins')
312
**kwargs: Overture query options (type, subtypes, etc.)
313
314
Returns:
315
gpd.GeoDataFrame: Overture data as GeoDataFrame
316
"""
317
318
def add_overture_layer(self, bbox, theme, **kwargs):
319
"""
320
Add Overture Maps layer to map.
321
322
Args:
323
bbox (list): Bounding box for data query
324
theme (str): Overture data theme
325
**kwargs: Layer styling and filtering options
326
"""
327
```
328
329
### TiTiler Integration
330
331
Integration with TiTiler for dynamic tiling of raster data from various sources.
332
333
```python { .api }
334
class TitilerEndpoint:
335
"""TiTiler endpoint for dynamic raster tiling."""
336
337
def __init__(self, endpoint='https://titiler.xyz'):
338
"""
339
Initialize TiTiler endpoint.
340
341
Args:
342
endpoint (str): TiTiler service URL
343
"""
344
345
def tile_url(self, url, **kwargs):
346
"""
347
Generate tile URL for raster.
348
349
Args:
350
url (str): Raster data URL
351
**kwargs: Tile parameters (rescale, colormap, etc.)
352
353
Returns:
354
str: Tile URL template
355
"""
356
357
def info(self, url):
358
"""
359
Get raster information.
360
361
Args:
362
url (str): Raster data URL
363
364
Returns:
365
dict: Raster metadata
366
"""
367
```
368
369
## Usage Examples
370
371
### STAC Data Discovery and Visualization
372
373
```python
374
import leafmap
375
376
# Search STAC catalog
377
items = leafmap.stac_search(
378
collection='landsat-c2-l2',
379
bbox=[-122.5, 37.7, -122.3, 37.8], # San Francisco
380
time_range=['2023-01-01', '2023-12-31'],
381
query={'eo:cloud_cover': {'lt': 10}}
382
)
383
384
print(f"Found {len(items)} items")
385
386
# Create map and add STAC layer
387
m = leafmap.Map(center=[37.75, -122.4], zoom=10)
388
389
if items:
390
m.add_stac_layer(
391
url=items[0]['assets']['rendered_preview']['href'],
392
name='Landsat Image'
393
)
394
395
m
396
```
397
398
### Microsoft Planetary Computer Workflow
399
400
```python
401
import leafmap
402
403
# List available collections
404
collections = leafmap.get_pc_collections()
405
print("Available collections:", collections[:5])
406
407
# Search for Sentinel-2 data
408
items = leafmap.pc_search(
409
collection='sentinel-2-l2a',
410
bbox=[-74.1, 40.7, -73.9, 40.8], # New York City
411
time_range=['2023-06-01', '2023-06-30']
412
)
413
414
# Create map and visualize
415
m = leafmap.Map(center=[40.75, -74.0], zoom=10)
416
417
if items:
418
m.add_stac_layer(
419
url=items[0]['self_href'],
420
assets=['B04', 'B03', 'B02'], # RGB bands
421
rescale='0,3000',
422
name='Sentinel-2 RGB'
423
)
424
425
m
426
```
427
428
### Planet Labs Integration
429
430
```python
431
import leafmap
432
433
# Search Planet catalog (requires API key)
434
items = leafmap.planet_search(
435
item_types=['PSScene4Band'],
436
bbox=[-122.5, 37.7, -122.3, 37.8],
437
date_range=['2023-07-01', '2023-07-31'],
438
cloud_cover=0.1
439
)
440
441
# Add Planet imagery
442
m = leafmap.Map(center=[37.75, -122.4], zoom=12)
443
444
if items:
445
m.add_planet_layer(
446
item_id=items[0]['id'],
447
item_type='PSScene4Band',
448
name='Planet Image'
449
)
450
451
m
452
```
453
454
### COG Visualization
455
456
```python
457
import leafmap
458
459
# COG from public dataset
460
cog_url = 'https://cloud.sdsc.edu/v1/AUTH_opentopography/Raster/SRTMGL1_Ellip_srtm/SRTMGL1_Ellip_srtm_09_05.tif'
461
462
# Get COG information
463
info = leafmap.cog_info(cog_url)
464
print("COG info:", info)
465
466
# Create map and add COG layer
467
m = leafmap.Map()
468
469
m.add_cog_layer(
470
url=cog_url,
471
name='SRTM Elevation',
472
colormap='terrain',
473
rescale='0,4000'
474
)
475
476
m
477
```
478
479
### AWS S3 Data Access
480
481
```python
482
import leafmap
483
484
# List objects in public bucket
485
objects = leafmap.s3_list_objects(
486
bucket='sentinel-s2-l2a',
487
prefix='tiles/32/T/NM/2023/7/15/',
488
max_keys=10
489
)
490
491
# Add S3-hosted raster
492
m = leafmap.Map()
493
494
if objects:
495
key = objects[0]['Key'] # First object
496
m.add_s3_layer(
497
bucket='sentinel-s2-l2a',
498
key=key,
499
name='Sentinel-2 from S3'
500
)
501
502
m
503
```
504
505
### Overture Maps Data
506
507
```python
508
import leafmap
509
510
# Get building data from Overture Maps
511
buildings = leafmap.get_overture_data(
512
bbox=[-122.42, 37.77, -122.41, 37.78], # Small area in SF
513
theme='buildings',
514
type='building'
515
)
516
517
print(f"Found {len(buildings)} buildings")
518
519
# Visualize on map
520
m = leafmap.Map(center=[37.775, -122.415], zoom=16)
521
522
m.add_gdf(
523
buildings,
524
layer_name='Overture Buildings',
525
style={'fillColor': 'blue', 'fillOpacity': 0.5}
526
)
527
528
m
529
```
530
531
## Authentication and Configuration
532
533
### API Keys and Tokens
534
535
```python
536
import os
537
538
# Set API keys as environment variables
539
os.environ['PLANET_API_KEY'] = 'your_planet_api_key'
540
os.environ['NASA_API_KEY'] = 'your_nasa_api_key'
541
os.environ['MAPBOX_TOKEN'] = 'your_mapbox_token'
542
543
# Or configure in code (not recommended for production)
544
import leafmap
545
leafmap.set_api_key('planet', 'your_api_key')
546
```
547
548
### STAC Endpoints
549
550
```python
551
# Common STAC endpoints
552
stac_endpoints = {
553
'planetary_computer': 'https://planetarycomputer.microsoft.com/api/stac/v1',
554
'earth_search': 'https://earth-search.aws.element84.com/v1',
555
'usgs_landsat': 'https://landsatlook.usgs.gov/stac-server',
556
'copernicus_dataspace': 'https://catalogue.dataspace.copernicus.eu/stac'
557
}
558
```
559
560
## Data Collections
561
562
### Microsoft Planetary Computer Collections
563
564
- `landsat-c2-l2`: Landsat Collection 2 Level 2
565
- `sentinel-2-l2a`: Sentinel-2 Level 2A
566
- `sentinel-1-rtc`: Sentinel-1 Radiometric Terrain Correction
567
- `modis`: MODIS satellite data
568
- `naip`: National Agriculture Imagery Program
569
- `aster-l1t`: ASTER Level 1T
570
- `cop-dem-glo-90`: Copernicus DEM Global 90m
571
572
### NASA Datasets
573
574
- MODIS Terra/Aqua products
575
- VIIRS Earth observation data
576
- GPM precipitation data
577
- SMAP soil moisture data
578
- ICESat-2 elevation data
579
580
### Planet Labs Item Types
581
582
- `PSScene4Band`: PlanetScope 4-band imagery
583
- `PSScene`: PlanetScope 3-band imagery
584
- `REScene`: RapidEye imagery
585
- `SkySatScene`: SkySat imagery
586
- `REOrthoTile`: RapidEye ortho tiles