pantheon-ai/bash-script-toolkit

Complete bash-script toolkit with generation and validation capabilities

Quality

97%

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Risky

Do not use without reviewing

GNU sed Reference Guide

Name: pantheon-ai/bash-script-toolkit
Rating: 0.97 (1 reviews)
Author: pantheon-ai

Overview

sed (Stream EDitor) is a powerful text processing tool that performs basic text transformations on an input stream (file or pipeline).

Official Manual: https://www.gnu.org/software/sed/manual/ Man Page: man sed

Basic Syntax

sed [OPTIONS] 'command' file
sed [OPTIONS] -e 'command1' -e 'command2' file
sed [OPTIONS] -f script.sed file

Common Options

-n, --quiet, --silent    # Suppress automatic output
-e SCRIPT               # Add script to commands
-f FILE                 # Add contents of FILE as commands
-i[SUFFIX]              # Edit files in-place
-r, -E                  # Use extended regular expressions
--debug                 # Annotate program execution

Basic Commands

Substitution (s)

# Basic substitution
sed 's/old/new/' file              # Replace first occurrence
sed 's/old/new/g' file             # Replace all occurrences
sed 's/old/new/2' file             # Replace second occurrence
sed 's/old/new/gi' file            # Case-insensitive, all

# With different delimiter
sed 's|/old/path|/new/path|g' file
sed 's#old#new#g' file

Deletion (d)

# Delete lines
sed '5d' file                       # Delete line 5
sed '5,10d' file                    # Delete lines 5-10
sed '/pattern/d' file               # Delete matching lines
sed '/^$/d' file                    # Delete empty lines
sed '/^#/d' file                    # Delete comment lines

Print (p)

# Print lines
sed -n '5p' file                    # Print only line 5
sed -n '5,10p' file                 # Print lines 5-10
sed -n '/pattern/p' file            # Print matching lines

Append (a), Insert (i), Change (c)

# Append after line
sed '5a\New line' file

# Insert before line
sed '5i\New line' file

# Change line
sed '5c\Replacement line' file

# With pattern
sed '/pattern/a\New line after match' file

Address Ranges

Line Numbers

sed '5s/old/new/' file              # Line 5 only
sed '5,10s/old/new/' file           # Lines 5-10
sed '5,$s/old/new/' file            # Line 5 to end
sed '1,5d' file                     # Delete first 5 lines

Patterns

sed '/start/,/end/d' file           # Delete from start to end pattern
sed '/pattern/s/old/new/' file      # Substitute in matching lines
sed '1,/pattern/d' file             # Delete from line 1 to first match

Special Addresses

sed '$d' file                       # Delete last line
sed '1d' file                       # Delete first line
sed '$s/old/new/' file              # Substitute in last line

Advanced Substitution

Backreferences

# Capture and reuse
sed 's/\([0-9]\+\)/Number: \1/' file

# Multiple captures
sed 's/\([a-z]\+\) \([0-9]\+\)/\2 \1/' file

# With ERE (-E or -r)
sed -E 's/([0-9]+)/Number: \1/' file
sed -E 's/([a-z]+) ([0-9]+)/\2 \1/' file

Flags

s/old/new/      # Replace first
s/old/new/g     # Replace all (global)
s/old/new/2     # Replace 2nd occurrence
s/old/new/i     # Case-insensitive
s/old/new/I     # Case-insensitive (same as i)
s/old/new/p     # Print if substitution made
s/old/new/w file # Write if substitution made

Special Characters in Replacement

&               # Matched string
\1, \2, etc     # Backreferences
\L, \U          # Convert to lower/upper (GNU sed)
\n              # Newline (in replacement)
\\              # Literal backslash

Multiple Commands

Multiple -e Options

sed -e 's/old/new/g' -e 's/foo/bar/g' file

Semicolon Separator

sed 's/old/new/g; s/foo/bar/g' file

Multi-line Script

sed '
s/old/new/g
s/foo/bar/g
/pattern/d
' file

In-place Editing

# Edit file in-place
sed -i 's/old/new/g' file

# Create backup
sed -i.bak 's/old/new/g' file

# Multiple files
sed -i 's/old/new/g' *.txt

Pattern Matching

BRE (Basic Regular Expressions) - Default

sed 's/^/#/' file                  # Add # at beginning
sed 's/$/;/' file                  # Add ; at end
sed 's/[0-9]\+/X/g' file          # Replace numbers (BRE)
sed 's/\<word\>/WORD/g' file      # Word boundaries (BRE)

ERE (Extended Regular Expressions)

sed -E 's/[0-9]+/X/g' file        # Replace numbers (ERE)
sed -E 's/(foo|bar)/baz/g' file   # Alternation
sed -E 's/\s+/ /g' file           # Multiple spaces to one

Practical Examples for Shell Scripts

Configuration File Editing

# Change a config value
sed -i 's/^Port .*/Port 2222/' /etc/ssh/sshd_config

# Uncomment a line
sed -i 's/^#\(.*option.*\)/\1/' config.file

# Comment out a line
sed -i 's/^\(.*dangerous.*\)/#\1/' config.file

# Add line after pattern
sed -i '/\[section\]/a new_setting = value' config.ini

Text Processing

# Remove trailing whitespace
sed 's/[[:space:]]*$//' file

# Remove leading whitespace
sed 's/^[[:space:]]*//' file

# Remove empty lines
sed '/^$/d' file

# Remove comments and empty lines
sed '/^#/d; /^$/d' file

# Double-space file
sed 'G' file

# Remove duplicate lines (consecutive)
sed '$!N; /^\(.*\)\n\1$/!P; D' file

Path Manipulation

# Change paths
sed 's|/old/path|/new/path|g' file

# Extract filename from path
echo "/path/to/file.txt" | sed 's|.*/||'

# Extract directory from path
echo "/path/to/file.txt" | sed 's|/[^/]*$||'

Log File Processing

# Extract IP addresses
sed -n 's/.*\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\).*/\1/p' log

# Filter by date
sed -n '/2025-01-01/,/2025-01-31/p' log

# Remove timestamp
sed 's/^[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\} [0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\} //' log

Code Refactoring

# Rename function
sed -i 's/\boldFunctionName\b/newFunctionName/g' *.sh

# Change variable
sed -i 's/\$old_var/\$new_var/g' script.sh

# Update shebang
sed -i '1s|^#!/bin/sh|#!/bin/bash|' *.sh

Advanced Features

Hold Space

# Hold space commands
h               # Copy pattern space to hold space
H               # Append pattern space to hold space
g               # Copy hold space to pattern space
G               # Append hold space to pattern space
x               # Exchange pattern and hold spaces

Hold Space Examples

# Reverse file
sed '1!G;h;$!d' file

# Print last line
sed -n '$p' file
sed -n 'h;$p' file

# Append next line to current
sed 'N;s/\n/ /' file

Branching

# Label and branch
:label          # Define label
b label         # Branch to label
t label         # Branch if substitution succeeded
T label         # Branch if substitution failed

Branching Examples

# Remove C-style comments
sed -n '
/\/\*/ {
    :loop
    /\*\// {
        s|/\*.*\*/||g
        p
        b
    }
    N
    b loop
}
p
' file

Common Patterns

Join Lines

# Join all lines
sed ':a;N;$!ba;s/\n/ /g' file

# Join lines ending with backslash
sed -e :a -e '/\\$/N; s/\\\n//; ta' file

Number Lines

sed = file | sed 'N;s/\n/\t/'

Reverse Lines

sed '1!G;h;$!d' file

Print Specific Lines

# Print line 5
sed -n '5p' file

# Print first 10 lines
sed -n '1,10p' file

# Print last 10 lines
sed -n -e :a -e '1,10!{P;N;D;};N;ba' file

Common Pitfalls in Shell Scripts

1. Special Characters in Pattern

# Wrong - . matches any character
sed 's/192.168.1.1/new/' file

# Right - escape dots
sed 's/192\.168\.1\.1/new/' file

# Or use different delimiter
sed 's|192.168.1.1|new|' file

2. Not Escaping Backreferences in BRE

# Wrong (BRE)
sed 's/([0-9]+)/\1/' file

# Right (BRE)
sed 's/\([0-9]\+\)/\1/' file

# Right (ERE)
sed -E 's/([0-9]+)/\1/' file

3. In-place Editing Without Backup

# Dangerous
sed -i 's/old/new/' important_file

# Safer
sed -i.backup 's/old/new/' important_file

4. Using sed for Line Counting

# Inefficient
sed -n '$=' file

# Better
wc -l < file

5. Not Quoting Variables Properly

# Wrong
sed "s/$old/$new/g" file  # Dangerous if vars contain /

# Better
old_escaped=$(printf '%s\n' "$old" | sed 's:[\\/&]:\\&:g')
new_escaped=$(printf '%s\n' "$new" | sed 's:[\\/&]:\\&:g')
sed "s/$old_escaped/$new_escaped/g" file

# Or use different delimiter
sed "s|$old|$new|g" file

Performance Tips

1. Use Appropriate Tools

# For simple replacements, consider using other tools
# sed is great for complex patterns, but for simple tasks:

# Instead of
sed 's/old/new/g' file

# Consider
tr 'old' 'new' < file  # For single-character replacement

2. Minimize Pattern Matching

# Less efficient
sed '/pattern/s/old/new/g' large_file

# More efficient if pattern is rare
grep 'pattern' large_file | sed 's/old/new/g'

3. Combine Commands

# Less efficient
sed 's/old/new/g' file | sed 's/foo/bar/g'

# More efficient
sed 's/old/new/g; s/foo/bar/g' file

Testing sed Commands

# Test before in-place edit
sed 's/old/new/g' file | head

# Show only changes
sed -n 's/old/new/gp' file

# Count changes
sed -n 's/old/new/gp' file | wc -l

Resources

tile.json

pantheon-ai/bash-script-toolkit

sed-reference.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}validator/references/

GNU sed Reference Guide

Overview

Basic Syntax

Common Options

Basic Commands

Substitution (s)

Deletion (d)

Print (p)

Append (a), Insert (i), Change (c)

Address Ranges

Line Numbers

Patterns

Special Addresses

Advanced Substitution

Backreferences

Flags

Special Characters in Replacement

Multiple Commands

Multiple -e Options

Semicolon Separator

Multi-line Script

In-place Editing

Pattern Matching

BRE (Basic Regular Expressions) - Default

ERE (Extended Regular Expressions)

Practical Examples for Shell Scripts

Configuration File Editing

Text Processing

Path Manipulation

Log File Processing

Code Refactoring

Advanced Features

Hold Space

Hold Space Examples

Branching

Branching Examples

Common Patterns

Join Lines

Number Lines

Reverse Lines

Print Specific Lines

Common Pitfalls in Shell Scripts

1. Special Characters in Pattern

2. Not Escaping Backreferences in BRE

3. In-place Editing Without Backup

4. Using sed for Line Counting

5. Not Quoting Variables Properly

Performance Tips

1. Use Appropriate Tools

2. Minimize Pattern Matching

3. Combine Commands

Testing sed Commands

Resources

sed-reference.mdvalidator/references/