Gulp Newer

Gulp Newer is a Gulp plugin that creates a transform stream to pass through only source files that are newer than their corresponding destination files. It optimizes build pipelines by intelligently skipping unchanged files, significantly reducing build times in projects with frequent incremental updates.

Package Information

• Package Name: gulp-newer
• Package Type: npm
• Language: JavaScript
• Installation: npm install gulp-newer --save-dev

Core Imports

const newer = require('gulp-newer');

Basic Usage

const gulp = require('gulp');
const newer = require('gulp-newer');
const imagemin = require('gulp-imagemin');

// Simple 1:1 source-to-destination mapping
gulp.task('images', function() {
  return gulp.src('src/img/**')
    .pipe(newer('build/img'))      // Only pass through newer images
    .pipe(imagemin())
    .pipe(gulp.dest('build/img'));
});

// Many:1 source-to-destination mapping (concatenation)
gulp.task('concat', function() {
  return gulp.src('lib/*.js')
    .pipe(newer('dist/all.js'))    // Pass through all if any are newer
    .pipe(concat('all.js'))
    .pipe(gulp.dest('dist'));
});

Architecture

Gulp Newer is built around Node.js streaming principles and integrates seamlessly with Gulp's vinyl file system:

• Transform Stream: Extends Node.js stream.Transform in object mode to process vinyl file objects
• Vinyl File Integration: Works with Gulp's vinyl file objects, accessing file paths and modification times via file.stat.mtime
• Asynchronous Processing: Uses promises (kew library) for non-blocking file system operations
• Lazy Evaluation: Only performs file system checks when files flow through the stream
• Memory Efficient: Buffers files only when necessary (many:1 scenarios) and clears buffers after processing

The plugin operates by comparing modification times between source files and their corresponding destination files, passing through only those source files that are newer than their destinations.

Capabilities

Main Function

Creates a transform stream that filters files based on modification time comparison.

/**
 * Creates a transform stream that passes through only newer source files
 * @param {string|NewerOptions} options - Destination path or options object
 * @returns {stream.Transform} Transform stream for file filtering
 */
function newer(options);

Parameters:

• options (string | NewerOptions): Either a destination path string or configuration object

Options Configuration

Configuration object for advanced filtering scenarios.

interface NewerOptions {
  /** Path to destination directory or file (required unless map is provided) */
  dest?: string;
  /** Extension for destination files (e.g., '.min.js') */
  ext?: string;
  /** Function to map source paths to destination paths */
  map?: (relativePath: string) => string;
  /** Extra files/globs to check for timestamp updates */
  extra?: string | string[];
}

Option Details:

• dest (string, required unless map provided): Path to destination directory or file
  • Directory path: Enables 1:1 source-to-destination mapping
  • File path: Enables many:1 source-to-destination mapping
• ext (string, optional): Override extension for destination files
• map (function, optional): Custom path mapping function for complex scenarios
• extra (string | Array<string>, optional): Additional dependency files that trigger rebuilds

Usage Patterns

1:1 Source-Destination Mapping

For scenarios where each source file corresponds to one destination file.

// Basic directory mapping
gulp.src('src/**/*.js')
  .pipe(newer('build'))           // Maps src/foo.js → build/foo.js

// Extension transformation
gulp.src('src/**/*.less')
  .pipe(newer({
    dest: 'build',
    ext: '.css'                   // Maps src/styles.less → build/styles.css
  }))
  .pipe(less())
  .pipe(gulp.dest('build'));

// Custom path mapping
gulp.src('src/**/*.ts')
  .pipe(newer({
    dest: 'dist',
    map: function(relativePath) {
      return relativePath.replace('.ts', '.js');
    }
  }))
  .pipe(typescript())
  .pipe(gulp.dest('dist'));

Many:1 Source-Destination Mapping

For scenarios where multiple source files contribute to a single destination file.

// Concatenation scenario
gulp.src('lib/*.js')
  .pipe(newer('dist/bundle.js'))  // All sources if any newer than bundle.js
  .pipe(concat('bundle.js'))
  .pipe(gulp.dest('dist'));

// Compilation with dependencies
gulp.src('templates/**/*.hbs')
  .pipe(newer('dist/templates.js'))
  .pipe(handlebars())
  .pipe(concat('templates.js'))
  .pipe(gulp.dest('dist'));

Extra Dependencies

Include additional files that should trigger rebuilds when modified.

gulp.src('src/**/*.scss')
  .pipe(newer({
    dest: 'build',
    ext: '.css',
    extra: [
      'config/variables.scss',    // Single extra file
      'mixins/**/*.scss'          // Glob pattern for multiple files
    ]
  }))
  .pipe(sass())
  .pipe(gulp.dest('build'));

// Multiple extra dependency patterns
gulp.src('src/**/*.js')
  .pipe(newer({
    dest: 'dist',
    extra: ['package.json', 'webpack.config.js', 'src/config/**']
  }))
  .pipe(webpack())
  .pipe(gulp.dest('dist'));

Error Handling

The plugin throws specific errors for invalid configurations:

// Error conditions that throw PluginError:
// - Missing options parameter
// - Invalid dest type (non-string when provided)
// - Invalid ext type (non-string when provided)  
// - Invalid map type (non-function when provided)
// - Missing both dest and map options
// - Invalid extra type (not string or array when provided)
// - File system errors when reading extra files

Common Error Scenarios:

// Throws: "Requires a dest string or options object"
newer();

// Throws: "Requires a dest string"
newer({ dest: 123 });

// Throws: "Requires either options.dest or options.map or both"
newer({});

// Throws: "Requires ext to be a string"
newer({ dest: 'build', ext: 123 });

// Throws: "Requires map to be a function"
newer({ dest: 'build', map: 'invalid' });

// Throws: "Requires options.extra to be a string or array"
newer({ dest: 'build', extra: 123 });

Implementation Notes

File Comparison Logic

• Modification Time: Compares stat.mtime between source and destination files
• Missing Destinations: Files with no corresponding destination are always passed through
• Extra Files: Any extra file newer than destination triggers rebuild of all sources
• Buffering: In many:1 scenarios, files are buffered until a newer file is found, then all are flushed

Performance Optimization

• Lazy Evaluation: File stats are checked only when needed
• Promise-based: Uses kew library for async file system operations
• Stream Processing: Integrates seamlessly with Gulp's vinyl file stream
• Minimal Overhead: Efficient stat checking without reading file contents

Stream Behavior

• Object Mode: Operates on vinyl file objects, not raw buffers, using {objectMode: true}
• Transform Stream: Extends Node.js stream.Transform class, inheriting proper backpressure handling and stream lifecycle
• Vinyl File Processing: Requires vinyl files with stat property containing file modification time (file.stat.mtime)
• Asynchronous Transform: Uses _transform(srcFile, encoding, done) method with callback-based completion
• Stream Flushing: Implements _flush(done) method to clean up buffered files and prevent memory leaks
• Error Propagation: File system errors are properly propagated through the stream using PluginError wrapper
• Buffering Strategy: In many:1 scenarios, files are buffered until a newer file is found, then all buffered files are flushed through the stream