Skip to main content

Saving data While it Travels Through the PowerShell Pipeline

A good question came up in class today while we were studying PowerShell Format commands.  We took a good look at how Format cmdlets consume the original object and make piping limited.  One of the questions that came up was if it is possible to save the information to a file and still be able to use it in the pipeline?  Fortunately for us, there is Tee-Object.
We have a couple of ways that we can implement Tee-Object.  Let’s look at it from a reusability perspective.  Here is our starting code:

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM

And here is the output:
Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

We will now add Tee-Object and send the data to another cmdlet.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    Tee-Object -FilePath C:\PS\Data1.txt |
    Sort-Object -Property VM

The object was successfully piped to Sort-Object and processed.  We can also read the text file that was created.

PS C:\ps> Get-Content data1.txt

Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

The reusability question now comes into play.  What if I needed to use the data that was sent to the text file in a later operation?  Well, it is now text.  That is not information that we could easily reincorporate into a PowerShell object.  Utilizing ConvertTo-CSV and ConvertFrom-CSV we can.  By placing ConvertTo-CSV before Tee-Object and ConvertFrom-CSV after Tee-Object was are able to send a comma separated file to disk.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    ConvertTo-CSV |
    Tee-Object -FilePath C:\PS\Data1.csv |
    ConvertFrom-CSV|
    Sort-Object -Property VM

Name     VM        PM      
----     --        --      
AcroRd32 132603904 10866688
AcroRd32 472272896 216985600
ApMsgFwd 96550912  1626112

The screen out shows the Sort-Object successfully worked with the object in the PowerShell pipeline.  Import-CSV is also able to successfully read the object back into member from disk.

PS C:\ps> Import-Csv -Path Data1.csv

Name     VM        PM      
----     --        --      
AcroRd32 472272896 216985600
AcroRd32 132603904 10866688
ApMsgFwd 96550912  1626112  


Let’s take a look at using the –Variable parameter of Tee-Object.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    Tee-Object -Variable Data |
    Sort-Object -Property VM

Here we removed the –FilePath parameter and added the –Variable parameter.  This is a formal usage of a variable so we omit the $ in front of the variable name. Here is the contents of the variable
.
PS C:\ps> $Data

Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

More importantly, here is the object:

PS C:\ps> $Data | Get-Member


   TypeName: Selected.System.Diagnostics.Process

Name        MemberType   Definition                   
----        ----------   ----------                   
Equals      Method       bool Equals(System.Object obj)
GetHashCode Method       int GetHashCode()            
GetType     Method       type GetType()               
ToString    Method       string ToString()            
Name        NoteProperty string Name=AcroRd32         
PM          NoteProperty long PM=216985600            
VM          NoteProperty long VM=472272896  

The information stored in $data is retained as an object.


So the question now comes up as to when to use the –FilePath parameter and when to use the –Variable parameter. If you will consume the data that was removed from the pipeline within the same scope of memory, use the –Variable parameter.  If you will consume the data in another scope of memory, such as another cmdlet or a different script, the use the –FilePath parameter

Comments

Popular posts from this blog

Adding a Comment to a GPO with PowerShell

As I'm writing this article, I'm also writing a customization for a PowerShell course I'm teaching next week in Phoenix.  This customization deals with Group Policy and PowerShell.  For those of you who attend my classes may already know this, but I sit their and try to ask the questions to myself that others may ask as I present the material.  I finished up my customization a few hours ago and then I realized that I did not add in how to put a comment on a GPO.  This is a feature that many Group Policy Administrators may not be aware of. This past summer I attended a presentation at TechEd on Group Policy.  One organization in the crowd had over 5,000 Group Policies.  In an environment like that, the comment section can be priceless.  I always like to write in the comment section why I created the policy so I know its purpose next week after I've completed 50 other tasks and can't remember what I did 5 minutes ago. In the Group Policy module for PowerShell V3, th

Return duplicate values from a collection with PowerShell

If you have a collection of objects and you want to remove any duplicate items, it is fairly simple. # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   # Remove the duplicate values. $Set1 | Select-Object -Unique 1 2 3 4 5 6 7 What if you want only the duplicate values and nothing else? # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   #Create a second collection with duplicate values removed. $Set2 = $Set1 | Select-Object -Unique   # Return only the duplicate values. ( Compare-Object -ReferenceObject $Set2 -DifferenceObject $Set1 ) . InputObject | Select-Object – Unique 1 2 This works with objects as well as numbers.  The first command creates a collection with 2 duplicates of both 1 and 2.   The second command creates another collection with the duplicates filtered out.  The Compare-Object cmdlet will first find items that are diffe

How to list all the AD LDS instances on a server

AD LDS allows you to provide directory services to applications that are free of the confines of Active Directory.  To list all the AD LDS instances on a server, follow this procedure: Log into the server in question Open a command prompt. Type dsdbutil and press Enter Type List Instances and press Enter . You will receive a list of the instance name, both the LDAP and SSL port numbers, the location of the database, and its status.