In which Patrick Index casts a jaundiced eye on SSIS, and decides that, for all its faults, it has a number of uses as an ETL tool. In the first of a series of articles 'from the trenches', Patrick describes how to pass variables to, and from, an SSIS task.
Passing Variables to and from an SSIS task
Like it? SSISht!
Like it or loath it, SSIS is here to stay. I suppose it’s nice and graphical, and it is also aesthetically pleasing when you finally get a screen full of green tasks – I tend to leave my screen displaying for a while when this happens, so that everyone can see it whilst I go and make a coffee. SSIS is much richer than DTS. Additionally you quite often see jobs for SSIS specialists; it would seem that companies are using it as their de-facto ETL tool, standing apart from SQL Server.
SSIS is, by its very nature, frustrating to work with because it is a mish-mash of dissimilar development environments, and I don’t find that the syntax is always intuitive
There doesn’t seem to be a great deal of material on the web and it can be hard to find good examples to use as guidelines. So, in the spirit of building up a knowledge base, and hopefully persuading Tony to set up a dedicated section on Simple-Talk for SSIS, I have constructed an example to demonstrate passing variables into and out of an ‘Execute SQL Task and Script Task’.
Passing Variables to and from an ‘Execute SQL Task and Script Task’.
The two tasks do fundamentally the same thing, which is to try and date-stamp a file. The final variable value “FullPath” could then be easily used by a File System Task to copy/move or delete a file perhaps.
I suppose most SQL Server developers would be more comfortable knocking up this fairly trivial code in SQL, but the difficulty is in passing and catching the input variables in the task. This example demonstrates the problem.
I have set up a package with three String variables called
- FileName, which has a data type of String and an initial value of “Import.txt”
- FolderName, which has a data type of String and an initial value of “c:\”
- FullPath, which has a data type of String and no initial value
… and an ‘Execute SQL Task and a Script’ Task.
The package is called, rather imaginatively, “Package3”. The scope of the variables is at the package level. One thing to note when you set up variables (choose SSIS-VARIABLES from the top menu) is to make sure you have clicked on the package and not a task when you create or add a variable. If you create a variable while being clicked on a task (therefore with task scope) then the variable will disappear from the list when you click up to the package. Sorry to be an old dog but I initially found this a bit confusing.
The simplest way to inspect your variables is to set a break-point on the task (right click on the task and choose EDIT BREAKPOINTS) for the OnPostExecute event of the task. This will then allow you to inspect the value of the variable after the task has completed its execution. The red dots on the tasks indicate that there are already breakpoints set up on the task.
Doing it the 'Execute SQL Task' way
In the ‘Execute SQL Task Editor’ in the ‘Parameter Mapping’ section, (double-click on the task and choose Parameter mapping), I have set it up so that the two variables i.e. User::FolderName and User::FileName are added to the dialogue box. Each has a Direction of “Input” which seems logical and a data type of VARCHAR. The parameter names that I have used are just the ordinal positions of 0 and 1, which is what the context help suggests. In other words, the value of User::FolderName is the first input parameter and User::FileName is the second input parameter. The parameter lengths are 50. In other words, we are dealing with a varchar(50) parameter. The initial default values for these, when you set them up, are -1 which tells me nothing I am afraid.
For the Result Set options on the Execute SQL Task, I have put in the aliased name of the output column from my query, which, giving my poetic instincts full rein, I have called FullPathFromQuery, and I want to pass this to my variable User::FullPath.
‘So what about the query?’ you may ask. Well, if we go back to the General option on the Execute SQL Task Editor, you can see the code and I will list it again here
DECLARE @YesterdaysDate varchar(8)
DECLARE @Filename varchar(50)
DECLARE @Folder varchar(50)
DECLARE @Etc varchar(50)
SET @Folder = ?
SET @Filename = ?
--SET @Etc = ?
SELECT @YesterdaysDate = CONVERT(varchar(8),DATEADD(dd,-1,getdate()),112)
SELECT @Folder + SUBSTRING(@Filename,1,CHARINDEX('.txt',@Filename)-1) + '_' + @YesterdaysDate + '.txt' AS FullPathFromQuery
For such trivial code you would not want to set up a stored procedure I suspect, so the option of passing parameters to a stored procedure is not really there.
The only way to pick up these input variable values is to use question marks “?” in the order that they are passed. This query as it stands will clearly not parse in query analyser window so you can only really test your code by running it in the task: This is not ideal.
You must also set the ResultSet option to be “Single row”.
If you run this to the break point one can see that the variable User::FullPath has been assigned the value C:\\Import_200870805.txt ( I ran this on 6th Aug 2008) so the code must be working.
Using a Script Task instead
Perhaps a simpler way to do this is just to use the Script Task. The trivial code again demonstrates how to pick up and assign values to and from variables in the vb.net code. You need to tell the task which variables it is going to use by adding them to the ReadOnlyVariables and ReadWriteVariables options in the Script Task editor window and also be aware that the variables names are case-sensitive.
The code which shows the passing and assignment of the variables in the script is shown below.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
Dim strFolder As String
Dim strFilename As String
Dim strTomorrow As String
Dim strNewFullPath As String
'do path in script
strFolder = Dts.Variables("FolderName").Value.ToString
strFilename = Dts.Variables("FileName").Value.ToString()
strTomorrow = CStr(Format(Today().AddDays(+1), "yyyyMMdd"))
strNewFullPath = strFolder & strFilename.Substring(0, strFilename.IndexOf(".txt")) & "_" & strTomorrow & ".txt"
'display new value of path
'MsgBox(strNewFullPath)
Dts.Variables("FullPath").Value = strNewFullPath
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
If you put a breakpoint on the task the value of the variable can be inspected to give tomorrows date, and it should look like this…
So which approach is best?
People tell me that using question marks was how it was in DTS, but I have to say I didn’t find setting up the Execute SQL Task particularly intuitive. The script task for me seems like the simplest solution of the two, surprisingly.
So the example is not going to win a Nobel Peace Prize for complexity but it does demonstrate the awkwardness of SSIS in the real world. Hopefully the article will encourage readers to publish their experiences using the various tasks and we can build up a knowledge base of real world examples for everyone to reference on the simple-talk site. I have recently been wrestling with the XML task and data source which will be the subject of my next article.