Andras

Software Architect - Red Gate Software

Pivots with Dynamic Columns in SQL Server 2005

Published Friday, September 14, 2007 9:28 AM

Pivots in SQL Server 2005 can rotate a table, i.e. they can turn rows into columns. PIVOTs are frequently used in reports, and they are reasonably simple to work with. However, I've seen quite a few questions about this operator. Most questions were about the column list in the PIVOT statement. This list is fixed, but many times the new columns are determined by the report at a later stage. This problem is easily solved when we mix pivots with dynamic SQL, so here is a very simple example about how to dynamically generate the pivot statement:

PIVOT allows you to turn data rows into columns. For example, if you have a table like this (I use only three months here for simplicity):

CREATE TABLE Sales ([Month] VARCHAR(20) ,SaleAmount INT)

INSERT INTO Sales VALUES ('January', 100)
INSERT INTO Sales VALUES ('February', 200)
INSERT INTO Sales VALUES ('March', 300)

SELECT FROM SALES
 

Month             SaleAmount
----------------  -----------
January           100
February          200
March             300 

Suppose we wanted to convert the above into this:

 
January     February    March
----------- ----------  ----------
100         200         300

We can do this using the PIVOT operator, as follows:

SELECT  [January]
      [February]
      [March]
FROM    SELECT    [Month]
                  SaleAmount
          FROM      Sales
        p PIVOT SUM(SaleAmount)
                    FOR [Month] 
IN ([January],[February],[March])                   ) AS pvt

However, in the above example, I have the column names fixed as the first three months. If I want to create a result in which the columns are dynamic (for example, they are read from a table), then I need to construct the above query dynamically. To demonstrate this let’s look at the following example:

In the first table I have the column names I want to use:

CREATE TABLE Table1 (ColId INT,ColName VARCHAR(10))
INSERT INTO Table1 VALUES(1, 'Country')
INSERT INTO Table1 VALUES(2, 'Month')
INSERT INTO Table1 VALUES(3, 'Day')

In the second table I have the data. This consists of a row identifier (tID), a column ID (ColID) that refers to the column type in Table1, and a value:

CREATE TABLE Table2 (tID INT,ColID INT,Txt VARCHAR(10))

INSERT INTO Table2 VALUES (1,1, 'US')
INSERT INTO Table2 VALUES (1,2, 'July')
INSERT INTO Table2 VALUES (1,3, '4')
INSERT INTO Table2 VALUES (2,1, 'US')
INSERT INTO Table2 VALUES (2,2, 'Sep')
INSERT INTO Table2 VALUES (2,3, '11')
INSERT INTO Table2 VALUES (3,1, 'US')
INSERT INTO Table2 VALUES (3,2, 'Dec')
INSERT INTO Table2 VALUES (3,3, '25')

Now I would like to retrieve data from these two tables, in the following format:


tID         Country    Day        Month
----------- ---------- ---------- ----------
1           US         4          July
2           US         11         Sep
3           US         25         Dec 

In other words I want to turn the data rows in Table2 into columns. If I had a fixed set of columns for the result, i.e. the columns Country, Day, and Month were fixed, I could use SQL Server 2005’s PIVOT operator in a query like:

SELECT  tID
      [Country]
      [Day]
      [Month]
FROM    SELECT    t2.tID
                  t1.ColName
                  t2.Txt
          FROM      Table1 AS t1
                    JOIN Table2 
AS t2 ON t1.ColId t2.ColID         p PIVOT MAX([Txt])                     FOR ColName IN [Country][Day],                                      [Month] ) ) AS pvt ORDER BY tID ;

However I need to construct this query dynamically, because the column names Country, Day, and Month are specified in a table, and can be changed independently from my query. In our case these columns are given in Table1.

In the first step to generate the final pivot query I need to create the list of columns, in this case [Country], [Day], [Month].

Since there is no string concatenation aggregator in SQL (a concatenation aggregator would not be deterministic without some order restriction), and since the column names are stored in rows of a table, I need to flatten these columns into a single row or variable. There are various solutions to achieve this. One solution would be to use a query like:

DECLARE @cols NVARCHAR(2000)
SELECT  @cols COALESCE(@cols ',[' colName ']',
                         '[' colName ']')
FROM    Table1
ORDER BY colName

This query works both on SQL Server 2000 and 2005. It is efficient, but some may not like it because it uses the same variable (@cols) on both sides of an assignment. Another solution that works on SQL Server 2005 only is to use XML PATH.

DECLARE @cols NVARCHAR(2000)
SELECT  @cols STUFF(( SELECT DISTINCT TOP 100 PERCENT
                                '],[' t2.ColName
                        FROM    Table1 AS t2
                        ORDER BY '],[' t2.ColName
                        FOR XML PATH('')
                      ), 12'') + ']'

This second query (I’ve seen this posted by Peter Larsson) has the advantage that it does not use the @cols variable on the right hand side. I like this solution more, since this can be extended as a general string concatenation aggregate in more complex queries.

Both of the above queries generate, from Table1, the string: ‘[Country],[Day], [Month]’. This column list is used twice in the pivot query that we aim to construct. Once it is use in the list of columns that we want to retrieve, and once it is used as the list of values that should become columns. Having constructed this list of columns above, we can just concatenate it with the missing parts of the pivot query like:

DECLARE @query NVARCHAR(4000)
SET @query N'SELECT tID, '+
@cols +'
FROM
(SELECT  t2.tID
      , t1.ColName
      , t2.Txt
FROM    Table1 AS t1
        JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( '+
@cols +' )
) AS pvt
ORDER BY tID;'

Executing this with

EXECUTE(@query)

will give us the expected result: a table that is pivoted and shows columns that were specified in a table:

tID         Country    Day        Month
----------- ---------- ---------- ----------
1           US         4          July
2           US         11         Sep
3           US         25         Dec
by András

Comments

 

Adam Machanic said:

I know this is just a touch on the anal-retentive side, but as currently written your queries are technically a vector for an injection attack (think of a malnamed column). To thwart this issue, please use the QUOTENAME function instead of concatenating square brackets on either side of the column name.

September 17, 2007 12:18 PM
 

András said:

There is no such thing like paranoia :). You are right Adam. While in my example the columns are not meant to come form a user directly, and I assumed it is "clean", there is no reason not to fortify code by default. Thanks for your comment.
 Andras
September 18, 2007 9:17 AM
 

Cheverton said:

Your approach is clever and instructive but isn't the following simpler?

DECLARE @query NVARCHAR(4000)

Select @query = N'SELECT tID
     , [1] as ['+ a.ColName + ']
     , [2] as ['+ b.ColName + ']
     , [3] as ['+ c.ColName + ']
FROM    ( SELECT    t2.tID
                 , t2.ColId
                 , t2.Txt
         FROM      Table2 t2
       ) p PIVOT ( MAX([Txt])
                   FOR ColId IN ( [1], [2], [3] ) ) AS pvt
ORDER BY tID ;'
From
Table1 a, Table1 b, Table1 c
where
a.ColId=1 and b.ColId=2 and c.ColId=3;

exec( @query)
September 18, 2007 9:41 AM
 

Cheverton said:

I needed to think 2 seconds more. Of course, your approach extends dynamically for any number of columns, not just any names of columns.
September 18, 2007 9:53 AM
 

» Pivot Tables with Dynamic Columns » Flixon.com » Blog Archive said:

October 29, 2007 4:53 PM
 

Praddumna said:

Hi
Can we insert a new column in our query result and have a default value to it.




December 1, 2007 12:34 AM
 

Pivots with Dynamic Columns in SQL Server 2005 « Quarksoft’s Notes said:

January 7, 2008 1:41 PM
 

Ferenc said:

Dear Andras!

For me isn’t working.
It is said The name ‘Select …’ is not a valid identifier.

When I copy the @query content SQL syntax, and run in a new query it’s completted succesfully.

Can you help me?
January 28, 2008 7:34 AM
 

suruchijain said:

Hi András!

Well I tried using ur tick, I guess the "pivot" keyword in the @query is not accepted by SQL Server 2000. the error I get on executing :

this is what I do :

DECLARE @cols NVARCHAR(2000)
SELECT  @cols = COALESCE(@cols + ',[' + colName + ']',
                        '[' + colName + ']')
FROM    Table1
ORDER BY colName
--print @cols

DECLARE @query NVARCHAR(4000)
SET @query = N'SELECT tID, '+
@cols +'
FROM
(SELECT  t2.tID
     , t1.ColName
     , t2.Txt
FROM    Table1 AS t1
       JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( '+
@cols +' )
) AS pvt
ORDER BY tID;'

exec @query

and the error I get is:

Server: Msg 203, Level 16, State 2, Line 26
The name 'SELECT tID, [Country],[Day],[Month]
FROM
(SELECT  t2.tID
     , t1.ColName
     , t2.Txt
FROM    Table1 AS t1
       JOIN Table2 AS t2 ON t1.ColId = t2.ColID) p
PIVOT
(
MAX([Txt])
FOR ColName IN
( [Country],[Day],[Month] )
) AS pvt
ORDER BY tID;' is not a valid identifier.

Could u plz tell me where I m going wrong ?
February 6, 2008 4:11 AM
 

Ferenc said:

Hi Suruchijain!

The solution is: Execute(@query) instead of exec @query!
February 7, 2008 1:54 AM
 

paulie said:

This is great, kudos to you sir! I've been searching for a solution like this for ages. This is a real enabler, thanks so much for sharing your knowledge.
March 1, 2008 2:03 PM
 

http://www.simple-talk.com/community/blogs/andras/archive/2007/09/14/37265.aspx said:

March 16, 2008 3:27 PM
 

thot said:

Hi!

This is a great article.

But i'm having problems to do this in my query.

I doing and quiz analysis and i want to do this dinamically.

i have 2 tables in the DB like this:
  -- Aswear (IDAsw, IDQuiz, IDOption, IDQuestion)
  -- Option (IDOption, IDQuestion, Item, Order)

And i want something like this:

Sex     | Yes | No
M        | 2    | 2
F         | 3    | 6

The column "Sex" have IDQuestion = 1 and the other question have only 2 option (Yes or No) and have the IDQuestion = 11

This is possible by using pivot??
Anyone can showme an example to get this??

Regards

April 15, 2008 12:24 PM
 

alangrn said:

Thanks. This is a great article. How do I wriet these results to a table? I am trying to get the results from the cross tab to be linked to another query.
July 3, 2008 7:36 AM
 

BRebey said:

Excellent article!  Thanks a million for all the work!  

If you ever discover a way to use dynamic columns without the size limitation of a local variable, please let me know!  This solution won't work for me, as my requirements are to support 2880 columns - i.e., two days of data samples taken at 1-minute intervals, with each time interval in a column.  

Thanks again!
August 2, 2008 12:58 AM
 

gopikrishna said:

Hi,Thanqs for the above..query..lot of useful but can we implement the same
withot putting the query in single quotes.
August 26, 2008 2:54 AM
 

gopikrishna said:

Thanqs andras.
For
Pivots with Dynamic Columns in SQL Server 2005
But can u give a solution for
PIVOT with dynamic columns but Without dynamic sql statements

bye.
reply me at:
krishnak70@yahoo.co.in
September 4, 2008 1:22 AM
 

JoeManJoe said:

I have a SQL SERVER 2005 table with 3 columns: Operator, Shift, and Date. Rows are like:
Operator    Shift   Date
John          Early


How can i use PIVOT to create a SQL SERVER 2005 report that lists the days of a month as columns and the o
October 26, 2008 12:42 PM
 

JoeManJoe said:

I have a SQL SERVER 2005 table with 3 columns: Operator, Shift, and Date. Rows are like:
Operator    Shift   Date
John          Early


How can i use PIVOT to create a SQL SERVER 2005 report that lists the days of a month as columns and the o
October 26, 2008 12:42 PM
 

JoeManJoe said:

I have a SQL SERVER 2005 table with 3 columns: Operator, Shift, and Date.
Sample rows are :
Operator    Shift      Date
John          Early      Oct. 1, 2008
Mary         Late       Oct. 1, 2008
John          Late       Oct. 2, 2008
Mary         Early      Oct. 2, 2008

How can i use PIVOT or Crosstab to create a SQL SERVER 2005 report that lists ALL the days of a month as columns and the operators as rows - one row for each operator. More importantly in the row-column intersection, I will like to see Shifts instead of aggregate data (as the shift field is non numeric).  I want something like this:

                 Oct 1         Oct 2     Oct 3    ..............................
John            Early         Late
Mary           Late          Early
October 26, 2008 12:55 PM
You need to sign in to comment on this blog

About András

András Belokosztolszki is the architect of SQL Compare versions 4, 5, 6 and 7, SQL Log Rescue and SQL Refactor. He is focused on database internals, database synchronization and database schema evolution.

















<September 2007>
SuMoTuWeThFrSa
2627282930311
2345678
9101112131415
16171819202122
23242526272829
30123456
A SysAdmin's Guide to Change Management
 In the first in a series of monthly articles, ‘Confessions of a Sys Admin’, Matt describes the issues... Read more...

Exchange: Recovery Storage Groups
 It can happen at any time: You get a request, as Admin, from your company, to provide the contents of... Read more...

Build Your Own Virtualized Test Lab
 Desmond Lee explains the fundamentals of building a fully functional test lab for Windows Servers and... Read more...

Rendering Hierarchical Data with the Treeview
 It sometimes happens that Web Server controls that visualize data don't quite fit with the way that... Read more...

SQL Server 2008: Performance Data Collector
 With Performance Data Collector in SQL Server 2008, you can now store performance data from a number of... Read more...