Optimizing SharePoint Online Migration Performance using PnP PowerShell

Migrating content to SharePoint Online can be challenging, especially when dealing with large datasets, throttling issues, and performance bottlenecks. To optimize migration performance, you can use PnP PowerShell to manage batch processing, reduce throttling, and enhance efficiency.

This guide provides step-by-step best practices and PowerShell scripts to optimize migration speed while ensuring data integrity.

Step 1: Preparing for Migration

1. Analyze Source Data

Before migrating, evaluate the size, structure, and complexity of your SharePoint or file share data.

$siteUrl = "https://yourtenant.sharepoint.com/sites/migration"
Connect-PnPOnline -Url $siteUrl -Interactive

# Get Library Storage Details
$lists = Get-PnPList | Where-Object { $_.BaseType -eq "DocumentLibrary" }
$lists | Select Title, ItemCount, Hidden, LastItemUserModifiedDate

✔ Identifies large document libraries that may slow down migration.

2. Increase Migration Throughput

Microsoft enforces API request limits per user. To bypass throttling:

Use multiple admin accounts
Perform migration in off-peak hours
Reduce migration batch size

Step 2: Using High-Performance Migration Approaches

1. Enable Scripting and Optimize Storage Quotas

Ensure that custom scripts are allowed and storage quotas are optimized.

# Enable custom scripting on SharePoint Online
Set-PnPTenantSite -Url $siteUrl -NoScriptSite:$false

# Increase storage quota for large migrations
Set-PnPTenantSite -Url $siteUrl -StorageQuota 102400  # 100 GB

✔ Prevents feature restrictions during migration.

2. Optimize Large File Uploads Using Parallel Processing

Batch uploads improve migration speed and avoid timeouts.

$sourcePath = "C:\MigrationFiles"
$targetLibrary = "Documents"

# Get all files in source directory
$files = Get-ChildItem -Path $sourcePath -File

# Upload files in parallel
$files | ForEach-Object -Parallel {
    $filePath = $_.FullName
    Add-PnPFile -Path $filePath -Folder $using:targetLibrary
} -ThrottleLimit 5

✔ Reduces upload time by processing files in parallel.

3. Perform Incremental Migrations

Instead of migrating everything at once, use incremental migration to transfer only new or modified files.

$modifiedSince = (Get-Date).AddDays(-7)  # Last 7 days
$incrementalFiles = Get-PnPListItem -List "Documents" | Where-Object {
    $_["Modified"] -gt $modifiedSince
}

foreach ($file in $incrementalFiles) {
    Move-PnPFile -SourceUrl $file["FileRef"] -TargetUrl "/sites/newlibrary/$($file["FileLeafRef"])"
}

✔ Transfers only recently modified files, improving efficiency.

Step 3: Handling Migration Throttling

1. Adjust Migration Speeds to Prevent Throttling

Throttling happens when Microsoft limits API requests. You can control request speed by adding delays.

$files | ForEach-Object {
    Add-PnPFile -Path $_.FullName -Folder "Documents"
    Start-Sleep -Seconds 5  # Add delay to prevent throttling
}

✔ Prevents Microsoft API limits from blocking migration.

2. Use Multiple Admin Accounts for High-Speed Migration

You can distribute migration jobs across multiple admin accounts.

$admins = @("admin1@yourtenant.onmicrosoft.com", "admin2@yourtenant.onmicrosoft.com")
$index = 0

foreach ($file in $files) {
    $admin = $admins[$index % $admins.Count]
    Connect-PnPOnline -Url $siteUrl -Credentials (Get-Credential -UserName $admin)
    Add-PnPFile -Path $file.FullName -Folder "Documents"
    $index++
}

✔ Prevents single-account throttling by using multiple admin identities.

Step 4: Verifying Migration Performance

1. Monitor Migration Status

After migration, track success rates, failed uploads, and large files.

$logFile = "C:\MigrationLog.csv"

$files | ForEach-Object {
    $fileUrl = "/sites/migration/Documents/$($_.Name)"
    $exists = Get-PnPFile -Url $fileUrl -ErrorAction SilentlyContinue

    [PSCustomObject]@{
        FileName = $_.Name
        Status   = if ($exists) { "Uploaded" } else { "Failed" }
    } | Export-Csv -Path $logFile -Append -NoTypeInformation
}

Write-Host "Migration Report Saved to $logFile"

✔ Generates a detailed log of all migrated files.

2. Validate Large File Transfers

Verify if large files were successfully migrated.

Get-PnPListItem -List "Documents" | Where-Object { $_["File_x0020_Size"] -gt 100MB } | Select Title, File_x0020_Size

✔ Ensures no large files were skipped.

Step 5: Automating Migration for Continuous Use

To automate recurring migrations, schedule a PowerShell task in Windows Task Scheduler.

1. Create a Scheduled Task

Use Task Scheduler to run migration scripts at non-peak hours.

$Action = New-ScheduledTaskAction -Execute "PowerShell.exe" -Argument "C:\Scripts\MigrationScript.ps1"
$Trigger = New-ScheduledTaskTrigger -Daily -At "2AM"
Register-ScheduledTask -TaskName "SharePointMigration" -Action $Action -Trigger $Trigger -User "youruser" -Password "yourpassword"

✔ Ensures automatic migrations without manual intervention.