Batch Operations

Batch Operations

Overview

This skill covers efficient techniques for bulk operations in ServiceNow:

• Mass record creation with proper relationships
• Bulk updates with validation and error handling
• Performance considerations for large datasets
• Transaction management and rollback strategies
• Parallel processing patterns

When to use: When creating or updating many records at once, data migrations, bulk imports, or mass data corrections.

Who should use this: Administrators, developers, and data migration specialists.

Prerequisites

• Roles: admin or table-specific write permissions
• Access: Target tables and related reference tables
• Knowledge: GlideRecord API, table relationships, ServiceNow data model
• Environment: Test in sub-production first for large operations

Performance Guidelines

Record Volume Recommendations

| Volume | Method | Estimated Time | |--------|--------|----------------| | 1-10 | Parallel MCP calls | 2 && gr.state == 1) { gr.escalation = 1; gr.worknotes = 'Auto-escalated: P2 older than 2 days without progress'; gr.update(); stats.p2updates++; } } }

gs.info('Conditional updates complete: ' + JSON.stringify(stats)); description: Apply conditional updates based on priority

### Phase 3: Performance Optimization

#### Step 3.1: Batch Processing Pattern

Process large datasets in manageable batches.

Tool: SN-Execute-Background-Script Parameters: script: | // Optimized batch processing var CONFIG = { table: 'incident', query: 'active=true', batchSize: 200, maxBatches: 50, pauseBetweenBatches: false // true adds 100ms delay };

var totalProcessed = 0; var batchNum = 0; var startTime = new GlideDateTime();

while (batchNum 0) { gs.error('Errors:\n' + JSON.stringify(results.errors, null, 2)); }

if (results.skipped.length > 0) { gs.warn('Skipped:\n' + JSON.stringify(results.skipped, null, 2)); } description: Batch update with comprehensive error tracking

#### Step 4.2: Retry Pattern

Implement retry logic for transient failures.

Tool: SN-Execute-Background-Script Parameters: script: | // Retry pattern for batch operations var MAXRETRIES = 3; var RETRYDELAY = 1000; // milliseconds

function updateWithRetry(tableName, sysId, data) { var attempts = 0; var lastError = null;

while (attempts = 0; i--) { var op = operations[i]; try { var rollback = new GlideRecord(op.table); if (rollback.get(op.sysid)) { rollback[op.field] = op.oldValue; rollback.setWorkflow(false); // Avoid triggering additional logic rollback.update(); gs.info('Rolled back: ' + op.table + ' ' + op.sysid); } } catch (rollbackError) { gs.error('Rollback failed for ' + op.sys_id + ': ' + rollbackError.message); } }

gs.info('Rollback complete'); } description: Batch operation with transaction-like rollback

### Phase 5: Data Migration Patterns

#### Step 5.1: Table-to-Table Migration

Migrate data between tables with transformation.

Tool: SN-Execute-Background-Script Parameters: script: | // Table-to-table migration var SOURCETABLE = 'xoldapprequests'; var TARGETTABLE = 'screquest'; var BATCH_SIZE = 100;

var fieldMapping = { 'oldnumber': 'number', 'olddescription': 'description', 'requestor': 'requestedfor', 'submitdate': 'opened_at' };

var migrated = 0; var errors = 0;

var source = new GlideRecord(SOURCETABLE); source.addQuery('migrated', false); source.setLimit(BATCHSIZE); source.query();

while (source.next()) { try { var target = new GlideRecord(TARGET_TABLE); target.initialize();

// Apply field mapping for (var oldField in fieldMapping) { var newField = fieldMapping[oldField]; target[newField] = source[oldField]; }

// Transform values target.state = mapState(source.oldstate); target.priority = mapPriority(source.oldpriority);

var newSysId = target.insert();

// Mark source as migrated source.migrated = true; source.migrated_to = newSysId; source.update();

migrated++;

} catch (e) { errors++; gs.error('Error migrating ' + source.old_number + ': ' + e.message); } }

gs.info('Migration complete: ' + migrated + ' migrated, ' + errors + ' errors');

function mapState(oldState) { var stateMap = { 'open': 1, 'in_progress': 2, 'closed': 3 }; return stateMap[oldState] || 1; }

function mapPriority(oldPriority) { var priorityMap = { 'critical': 1, 'high': 2, 'medium': 3, 'low': 4 }; return priorityMap[oldPriority] || 4; } description: Table-to-table data migration with transformation

#### Step 5.2: External Data Import

Import data from external source (JSON array).

Tool: SN-Execute-Background-Script Parameters: script: | // External data import pattern var importData = [ { email: 'user1@company.com', firstname: 'John', lastname: 'Doe', department: 'IT' }, { email: 'user2@company.com', firstname: 'Jane', lastname: 'Smith', department: 'HR' }, { email: 'user3@company.com', firstname: 'Bob', lastname: 'Wilson', department: 'Finance' } ];

var results = { created: 0, updated: 0, errors: 0 };

importData.forEach(function(data) { try { // Check for existing record var existing = new GlideRecord('sys_user'); existing.addQuery('email', data.email); existing.query();

if (existing.next()) { // Update existing existing.firstname = data.firstname; existing.lastname = data.lastname; existing.department = lookupDepartment(data.department); existing.update(); results.updated++; gs.info('Updated user: ' + data.email); } else { // Create new var newUser = new GlideRecord('sysuser'); newUser.initialize(); newUser.email = data.email; newUser.username = data.email.split('@')[0]; newUser.firstname = data.firstname; newUser.lastname = data.lastname; newUser.department = lookupDepartment(data.department); newUser.active = true; newUser.insert(); results.created++; gs.info('Created user: ' + data.email); }

} catch (e) { results.errors++; gs.error('Error importing ' + data.email + ': ' + e.message); } });

gs.info('Import complete: ' + JSON.stringify(results));

function lookupDepartment(name) { var dept = new GlideRecord('cmndepartment'); dept.addQuery('name', name); dept.query(); if (dept.next()) { return dept.sysid.toString(); } return ''; } description: Import external user data with upsert logic

## Tool Usage Summary

| Operation | MCP Tool | Purpose |
|-----------|----------|---------|
| Batch Create | SN-Batch-Create | Create multiple records |
| Batch Update | SN-Batch-Update | Update multiple records by sys_id |
| Parallel Create | SN-Create-Record (multiple) | Maximum throughput |
| Parallel Update | SN-Update-Record (multiple) | Maximum throughput |
| Complex Batch | SN-Execute-Background-Script | Advanced logic |
| Query | SN-Query-Table | Find records for batch operations |

## Performance Comparison

| Method | 100 Records | 1,000 Records | 10,000 Records |
|--------|-------------|---------------|-----------------|
| Serial MCP calls | ~50 sec | ~500 sec | Not practical |
| Parallel MCP (10) | ~5 sec | ~50 sec | ~500 sec |
| SN-Batch-Create | ~3 sec | ~30 sec | ~300 sec |
| Background script | ~2 sec | ~20 sec | ~200 sec |
| setWorkflow(false) | ~1 sec | ~10 sec | ~100 sec |

## Best Practices

- **Always Test First:** Run with DRY_RUN=true or on small subset
- **Use Limits:** Never process unlimited records
- **Batch Appropriately:** 100-500 records per batch for optimal performance
- **Log Progress:** Report progress every 10-100 operations
- **Handle Errors:** Implement try-catch for each record
- **Track Changes:** Store old values for potential rollback
- **Consider Business Rules:** Use setWorkflow(false) only when appropriate
- **Monitor Performance:** Track processing rate (records/second)
- **Use Transactions:** Group related changes for atomic operations
- **Document Operations:** Log what was changed for audit purposes

## Troubleshooting

### Operation Timeouts

**Symptom:** Script execution stops mid-process
**Causes:**
- Too many records in single execution
- Complex business rules triggered
**Solution:**
- Reduce batch size
- Use self-scheduling pattern
- Disable workflows if appropriate

### Memory Errors

**Symptom:** "Out of memory" or slow performance
**Causes:**
- Loading too many records at once
- Storing results in arrays
**Solution:**
- Use chooseWindow() for pagination
- Process and discard immediately
- Avoid accumulating results in memory

### Duplicate Records Created

**Symptom:** Multiple copies of same record
**Causes:**
- Missing unique constraint check
- Script ran multiple times
**Solution:**
- Always check for existing before insert
- Use unique keys for deduplication
- Add idempotency checks

### Business Rules Not Firing

**Symptom:** Expected side effects not occurring
**Causes:**
- setWorkflow(false) in use
- Record updated incorrectly
**Solution:**
- Remove setWorkflow(false) if rules are needed
- Verify field updates are valid
- Check business rule conditions

## Related Skills

- `admin/script-execution` - Background script execution
- `admin/update-set-management` - Track batch operation changes
- `admin/user-provisioning` - Bulk user operations
- `admin/deployment-workflow` - Deploy batch scripts between instances

## References

- [ServiceNow GlideRecord](https://developer.servicenow.com/dev.do#!/reference/api/utah/server/c_GlideRecordScopedAPI)
- [GlideAggregate](https://developer.servicenow.com/dev.do#!/reference/api/utah/server/c_GlideAggregateScopedAPI)
- [Import Sets](https://docs.servicenow.com/bundle/utah-platform-administration/page/administer/import-sets/concept/c_ImportSets.html)
- [Performance Best Practices](https://developer.servicenow.com/dev.do#!/guides/utah/now-platform/tpb-guide/scripting_technical_best_practices)

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Happy-Technologies-LLC](https://github.com/Happy-Technologies-LLC)
- **Source:** [Happy-Technologies-LLC/happy-platform-skills](https://github.com/Happy-Technologies-LLC/happy-platform-skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.