, , ,

I’ve revisited my Twitch stream stats collector script.  Previously, it could only cope with one stream on a given day.  I’ve rehashed it so this is no longer an issue.  It still expects you to have only one stream’s worth of log files present in the mIRC log folder though. First here’s the code. Once again, I had to alter the quotes to get WordPress to do the highlighting correctly.

$strums = 'monstercat'

Function Parse-mIRC {
    #requires -Version 3
    BEGIN {
        $logRootPath = "$($env:APPDATA)\mIRC\Logs"
        #eg [11:41] * nightbot (nightbot@nightbot.tmi.twitch.tv) has joined #monstercat'
        $regex = "^\[[0-9][0-9]:[0-9][0-9]\] \* (?<Username>[A-Za-z0-9_\-]+) \(" # User joins or parts channel

    PROCESS  {
        $channels | ForEach-Object {
    # Init
            $channel = $_
            $databasePath = Join-Path -Path $databaseFolder -ChildPath "mIRC-$($channel).db"
            $streamViewers = New-Object -TypeName System.Collections.ArrayList -ArgumentList 64 # Initial capacity
            $imported = 0
            $added = 0
            $updated = 0

    # Create/import database
            if ((Test-Path -Path $databasePath) -eq $False) {
                Write-Verbose "*** Creating new database $databasePath ***"
                New-Item -Path $databasePath -ItemType File -Value '' | Out-Null
            } else {
                Write-Verbose "*** Importing database ***"
            $fromJson = Get-Content -Path $databasePath | ConvertFrom-Json
            $viewers = @{}
            if ($fromJson -ne $null) {
                $fromJson | ForEach-Object {
                    $viewers.Add($_.Username, $_)
                    Write-Verbose ("Importing user '{0}'..." -f $_.Username)
                    $imported += 1
            Write-Verbose ("Imported {0} users" -f $imported)

    # Get stream's viewers
            Write-Verbose "`n*** Finding this streams viewers ***"
            [Array]$logFiles = Get-ChildItem -Path "$logRootPath\#$($channel).*.log"
            if ($logFiles.Count -gt 0) {
                $streamDate = $logFiles[0].CreationTime.ToString("dd/MM/yyyy")
                $logContents = Get-Content -Path $logFiles # Concatenates them all
                foreach ($line in $logContents) {
                    if ($line -match $regex) {
                        $username = $matches.Username.ToLower()
                        if ($streamViewers -notcontains $username) {
                            [void]($streamViewers.Add($username)) # Cast is faster than Out-Null
    # Compare today's viewers to historical data
                Write-Verbose "`n*** Analysing this streams viewers ***"
                foreach ($username in $streamViewers) {
                    if ($viewers.ContainsKey($username) -eq $False) {
                        Write-Verbose ("Adding new user '{0}'..." -f $username)
                        $props = @{
                            Username = $username;
                            FirstSeen = $streamDate;
                            LastSeen = $streamDate;
                            StreamCount = 1;
                            Comments = ''
                        $newViewer = New-Object -TypeName PSObject -Property $props
                        $viewers.Add($username, $newViewer)
                        $added += 1   
                    } else {
                        Write-Verbose ("Updating user '{0}'..." -f $username)
                        if ($viewers[$username].LastSeen -ne $streamDate) {
                            $viewers[$username].LastSeen = $streamDate
                        $viewers[$username].StreamCount = $viewers[$username].StreamCount + 1
                        $updated += 1
                } # Today's Viewers
            } # Logs present
    # Write updated database
            Write-Verbose "`n*** Updating database ***"
            $viewers.Keys | Sort-Object | ForEach-Object {
                $viewers[$_] | ConvertTo-Json -Compress
            } | Set-Content -Path $databasePath -Force
            Write-Verbose ("`nAdded {0} and updated {1} viewers from {2} log files." -f $added, $updated, $logFiles.Count)
        } # Channels

    END {}
Parse-mIRC -databaseFolder "C:\Twitch\" -channels $strums -Verbose

For reference, here are the compatible mIRC logging settings for the RegEx I’ve used:
mIRC Log Settings

The script now parses the log files and gets a list of all names who appear in them. It then goes through that list to update the database. To build the list of viewers, I could have created a normal empty array and then added new strings of usernames to it as they are found in the log. Unfortunately, every time you add to an array, a new array object is actually created. If you had loads of viewers, this inefficient process could be problematic. I’ve therefore used a .NET type called an ArrayList.

An ArrayList is created with an initial “Capacity”. This is the sort-of-maximum size and not to be confused with the Count (the current number of elements). Once you’ve added elements past this capacity, it automatically creates a new object with a bigger capacity. You can just keep adding new elements and it will resize it (make a bigger replacement object) as required. You can add elements with a “+=” or you can use the .Add() method of the object. Rather than casting a new variable directly to the type, I used the New-Object command so I could pass an integer to the constructor of the initial capacity, just to makes things a bit more efficient. Here are the relevant bits of code:

$streamViewers = New-Object -TypeName System.Collections.ArrayList -ArgumentList 64

if ($streamViewers -notcontains $username) {

Note that when you use the .Add() method, you a returned an integer containing the index at which the element was added. I could have got rid of this with a pipe to Out-Null but this is quite slow, despite feeling more PowerShelly and less C-Sharpy! Instead, I cast the result to “[void]” to get rid of it as this is significantly quicker and given that this would be happening a lot, I decided it was worth the odder syntax.