Collection (TA0009)

Sub-technique: T1119 - Automated Collection

Objective: Detect automated collection of data for exfiltration.

1. Identify Automated File Collection

DeviceFileEvents
| where FileName has_any ("robocopy", "xcopy", "copy")
| project Timestamp, DeviceName, FileName, FolderPath, InitiatingProcessAccountName, InitiatingProcessFolderPath
| order by Timestamp desc

// Extended Search
DeviceFileEvents
| where FileName has_any ("robocopy", "xcopy", "copy")
| summarize FileCopyCount = count() by DeviceName, FileName
| join kind=leftouter (
    DeviceProcessEvents
    | where ProcessCommandLine has_any ("robocopy", "xcopy", "copy")
    | summarize ProcessCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, FileName, FileCopyCount, ProcessCount, ConnectionCount
| order by FileCopyCount desc

Purpose: Detect automated file copying commands.

1. Detection of Large Data Archives

DeviceFileEvents
| where FileName endswith ".zip" or FileName endswith ".rar"
| project Timestamp, DeviceName, FileName, FolderPath
| order by Timestamp desc

//More expanded search
DeviceFileEvents
| where FileName endswith ".zip" or FileName endswith ".rar"
| summarize ArchiveFileCount = count() by DeviceName, FileName, FolderPath
| join kind=leftouter (
    DeviceProcessEvents
    | where ProcessCommandLine has_any ("zip", "rar")
    | summarize ProcessCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, FileName, FolderPath, ArchiveFileCount, ProcessCount, ConnectionCount
| order by ArchiveFileCount desc

Purpose: Monitor the creation of large archive files.

1. Suspicious Data Collection Scripts

DeviceProcessEvents
| where ProcessCommandLine has_any ("backup", "sync", "archive")
| project Timestamp, DeviceName, ProcessCommandLine
| order by Timestamp desc

//More expansive search
DeviceProcessEvents
| where ProcessCommandLine has_any ("backup", "sync", "archive")
| summarize ProcessCount = count() by DeviceName, ProcessCommandLine
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".zip" or FileName endswith ".rar"
    | summarize ArchiveFileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, ProcessCommandLine, ProcessCount, ArchiveFileCount, ConnectionCount
| order by ProcessCount desc

Purpose: Detect scripts or commands used for data collection.

1. Detect Collection of Network Traffic Data

DeviceProcessEvents
| where ProcessCommandLine has_any ("tcpdump", "wireshark", "netsh")
| project Timestamp, DeviceName, ProcessCommandLine, AccountName
| order by Timestamp desc

//Extended search
DeviceProcessEvents
| where ProcessCommandLine has_any ("tcpdump", "wireshark", "netsh")
| summarize ProcessCount = count() by DeviceName, ProcessCommandLine
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".pcap" or FileName endswith ".cap"
    | summarize FileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, ProcessCommandLine, ProcessCount, FileCount, ConnectionCount
| order by ProcessCount desc

Purpose: Identify network traffic data collection.

1. Monitor for Data Collection via PowerShell

DeviceProcessEvents
| where ProcessCommandLine has "powershell" and ProcessCommandLine has_any ("Out-File", "Export-Csv")
| project Timestamp, DeviceName, ProcessCommandLine, AccountName
| order by Timestamp desc

//Extended Search
DeviceProcessEvents
| where ProcessCommandLine has "powershell" and ProcessCommandLine has_any ("Out-File", "Export-Csv")
| summarize ProcessCount = count() by DeviceName, ProcessCommandLine
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".csv" or FileName endswith ".txt"
    | summarize FileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, ProcessCommandLine, ProcessCount, FileCount, ConnectionCount
| order by ProcessCount desc

Purpose: Detect PowerShell commands used to export data.

1. Detect Database Dumps

DeviceProcessEvents
| where ProcessCommandLine has_any ("mysqldump", "pg_dump", "mongodump")
| project Timestamp, DeviceName, ProcessCommandLine
| order by Timestamp desc
 
 //Extended Search
 DeviceProcessEvents
| where ProcessCommandLine has_any ("mysqldump", "pg_dump", "mongodump")
| summarize ProcessCount = count() by DeviceName, ProcessCommandLine
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".sql" or FileName endswith ".dump"
    | summarize FileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3306 or RemotePort == 5432 or RemotePort == 27017
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, ProcessCommandLine, ProcessCount, FileCount, ConnectionCount
| order by ProcessCount desc

Purpose: Identify database dump commands.

1. Monitor for Automated Collection via Scripts

DeviceProcessEvents 
| where ProcessCommandLine has_any (".bat", ".ps1", ".sh") and ProcessCommandLine has_any ("copy", "export", "backup") 
| project Timestamp, DeviceName, FileName, AccountName, ProcessCommandLine, InitiatingProcessCommandLine, InitiatingProcessFileName, InitiatingProcessFolderPath

//Extended Search
DeviceProcessEvents
| where ProcessCommandLine has_any (".bat", ".ps1", ".sh") and ProcessCommandLine has_any ("copy", "export", "backup")
| summarize ProcessCount = count() by DeviceName, ProcessCommandLine
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".zip" or FileName endswith ".rar"
    | summarize ArchiveFileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, ProcessCommandLine, ProcessCount, ArchiveFileCount, ConnectionCount
| order by ProcessCount desc

Purpose: Detect scripts used for data collection.

1. Identify Collection of Sensitive Files

DeviceFileEvents
| where FileName has_any ("passwords.txt", "confidential.docx")
| project Timestamp, DeviceName, FileName, FolderPath, InitiatingProcessAccountName
| order by Timestamp desc

//Extended Search
DeviceFileEvents
| where FileName has_any ("passwords.txt", "confidential.docx")
| summarize FileAccessCount = count() by DeviceName, FileName, FolderPath, InitiatingProcessAccountName
| join kind=leftouter (
    DeviceProcessEvents
    | where ProcessCommandLine has_any ("copy", "move", "delete")
    | summarize ProcessCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceNetworkEvents
    | where RemotePort == 3389
    | summarize ConnectionCount = count() by DeviceName
) on DeviceName
| project DeviceName, FileName, FolderPath, InitiatingProcessAccountName, FileAccessCount, ProcessCount, ConnectionCount
| order by FileAccessCount desc

Purpose: Monitor access to sensitive files.

1. Detect Use of Cloud Services for Data Collection

DeviceNetworkEvents 
| where RemoteIP in ("cloud_storage_ip_list") 
| summarize count() by RemoteIP, LocalIP 
| where count() > 10

//Extended Search
DeviceNetworkEvents
| where RemoteIP in ("cloud_storage_ip_list")
| summarize ConnectionCount = count() by RemoteIP, DeviceName
| where ConnectionCount > 10
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".zip" or FileName endswith ".rar"
    | summarize ArchiveFileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceProcessEvents
    | where ProcessCommandLine has_any ("upload", "sync", "backup")
    | summarize ProcessCount = count() by DeviceName
) on DeviceName
| project RemoteIP, DeviceName, ConnectionCount, ArchiveFileCount, ProcessCount
| order by ConnectionCount desc

_Purpose_: Monitor data collection via cloud services.

10. Monitor for Data Collection via Network Shares

DeviceNetworkEvents 
| where RemotePort == 445 
| summarize count() by RemoteIP, LocalIP 
| where count() > 20

//Extended Search
DeviceNetworkEvents
| where RemotePort == 445
| summarize ConnectionCount = count() by RemoteIP, DeviceName
| where ConnectionCount > 20
| join kind=leftouter (
    DeviceFileEvents
    | where FileName endswith ".zip" or FileName endswith ".rar"
    | summarize ArchiveFileCount = count() by DeviceName
) on DeviceName
| join kind=leftouter (
    DeviceProcessEvents
    | where ProcessCommandLine has_any ("copy", "move", "delete")
    | summarize ProcessCount = count() by DeviceName
) on DeviceName
| project RemoteIP, DeviceName, ConnectionCount, ArchiveFileCount, ProcessCount
| order by ConnectionCount desc

Purpose: Identify data collection via network shares.