My PASS Summit 2013 Presentations

I had the honor to speak for the first time at this past PASS Summit 2013 in Charlotte, NC. It was a great opportunity for me as this is one of the most prestigious nationwide conferences on SQL Server. I delivered 3 different sessions on Business Intelligence related topics. Attendance to my sessions exceeded my expectations and my overall experience as a first-time speaker was very positive. I also need to add that the God’s of the live demos were on my side.

My session titled PowerPivot + Power View = Self-service BI Revolution was full and had about 5 to 8 people standing in the back. Everybody got excited when I did a demo of the nice interactivity in Power View and 3D geographical capabilities of PowerMap.

The Lightning Talk room was packed and people had to be turned away. This room should be bigger next time as these Lightning Talks are very popular. Everybody accepted and agreed with my postulation on the need for Conformed Stored Procedures in addition to Conformed Dimensions and Facts. After the Lightning Talks were over, several attendees came forward to discuss further how Conformed Stored Procedures should be implemented. One attendee in particular, mentioned that he had been thinking about proposing a similar approach to his team and this talk helped him get re-assurance of this approach.

My last session on SSAS Hierarhcies Tips & Tricks was a half-day session on the last day of the summit between 2:00pm and 5:00pm. It was half-full for the first half of the session, and ended up about 1/4 full.  This drop-off is expected towards the end of the Summit as many people fly back home on Friday and usually leave around 4:00pm.

I’m very pleased with the turnout and feedback received. The audience in all of my sessions was very engaged and had a lot of great questions. I definitely look forward to speak at PASS Summit 2014 in Seattle, WA and at PASS BA Conference 2014 in San Jose, CA.

To download my presentations slides click on the links below:

1. Regular Session (75 minutes): 4798 PowerPivot + Power View = Self-Service BI Revolution
2. Lightning Talk Session (10 minutes): Slides: 5511 Conformed Stored Procedures Script: ConformedStoredProcs_SQLScript
3. Half-Day Session (3 hours): 4803 Fast Performing SSAS Hierarchies Tips and Tricks

Thanks to all the attendees and PASS team for this great opportunity. I hope I met or exceeded everyone’s expectations.

If you attended one of my presentations, please leave your comments and feedback!

.Net Code Camp Orlando 2013

This past Saturday March 16th, I had the opportunity to speak once again the .Net Orlando Code Camp.  I presented a session under the SQL Server track titled “Change Data Capture, a developer’s best unknown friend”.  I had better than expected turn-out with about 20 attendees and overall very good participation.

I was glad to hear from more than one attendee that they were planning to use Change Data Capture in their environment and best of all to get rid of triggers!

I had the opportunity to catch-up with some of my friends and fellow SQL-ers from PASS. Also, met some new folks from the App/Dev community. These type of events are always great to expand your network, share your knowledge and learn something new!

Thanks to the organizers, sponsors, volunteers and attendees to make this event a success and for having me once more.

You can download my PowerPoint presentation and demo files at: http://sdrv.ms/15UUX64

Afterthoughts: IT Pro Camp Orlando 2012

This past Saturday January 21, 2012 I had the opportunity to speak at my first IT Pro Camp in Orlando, FL. This event was put together by Blain Barton ( Blog | LinkedIn | Twitter ), Microsoft Senior IT Pro Evangelist and Chad Miller ( Blog | LinkedIn | Twitter ), Microsfot MVP and Senior Manager of Database Administration at Raymond James Financial.

Event

The event went very smooth and had a good turnaout. Food was amazing. They reallly outdid themselves with good old southern BBQ.

Presentation

I presented a session titled Introduction to Microsoft Business Intelligence (Slides), focusing on the basic concepts of BI and Datawarehousing, the Microsoft toolset and my Top 5 Rules of Wisdom for succesful BI. I had a great turnaout, about 25 attendees. All of the attendees were very engaged and asked excellent questions. I enjoyed presenting to the Orlando crowd.

Picture credits: Russel Faustino. See more pictures of the event here.

Other IT Pro Camps are being added. Check the IT Pro Camp website here to keep up to date.

Click on the picture below to download my Intorduction to Business Intelligence powerpoint presentation.

 

 

Codeplex: SSIS Community Tasks and Components

I recently bumped into this very exhaustive list of SSIS tasks, components and samples in CodePlex http://ssisctc.codeplex.com. The list is maintained by SQL Server MVP (Blog|Twitter) and  grouped in the following sections:

• Tools
• Connection Managers
•  Log Providers
• Tasks (for Control Flow)
• Foreach Enumerators
• Script Task
• Script Samples
• Components (for Data Flow)
• Script Component Samples
•  Sources
• Transforms
• Destinations

I hope this list keeps being updated and more people contribute. Very useful.

The CASE of the missing non-NULL T-SQL Error: None of the result expressions in a CASE specification can be NULL.

Recently, while doing some data scrubbing for a customer I got an interesting error in SSMS with one of my CASE statements. The error received was:

None of the result expressions in a CASE specification can be NULL.

It was a long T-SQL script (and a long night) and could not understand the error message. After a couple minutes BinGling (Google +Bing) around the web, I still could not find the root cause, so I decided to comment out every single CASE statement and run one by one until I pinpointed the offending syntax.

 

To my surprise, the issue was very simple, yet undocumented. The T-SQL syntax I was writing was somewhat as follows:

 

SELECT ProductID,

CASE WHEN SerialNumber like ‘X%’ THEN NULL

WHEN SerialNumber = ‘0’ THEN NULL

WHEN SerialNumber = ‘-‘ THEN NULL

WHEN SerialNumber = ‘Unknown’ THEN NULL

END AS SerialNumber_Clean

FROM tblProduct

 

After executing this script the error mentioned above is raised. The same error is raised even if we rewrite the script as a simple CASE statement:

 

SELECT ProductID,

CASE SerialNumber

WHEN ‘0’ THEN NULL

WHEN  ‘-‘ THEN NULL

WHEN ‘Unknown’ THEN NULL

END AS SerialNumber_Clean

FROM tblProduct

 

So what if we add an ELSE statement as follows:

 

SELECT ProductID,

CASE WHEN SerialNumber like ‘X%’ THEN NULL

WHEN SerialNumber = ‘0’ THEN NULL

WHEN SerialNumber = ‘-‘ THEN NULL

WHEN SerialNumber = ‘Unknown’ THEN NULL

ELSE NULL

END AS SerialNumber_Clean

FROM tblProduct

 

We still get the same error. The issue as the raised error describes can be narrowed down in that you cannot explicitly return NULL for every resulting expression. There must be at least one non-explicit NUL in the resulting expression. For example, we can rewrite the script correctly as follows:

 

SELECT ProductID,

CASE WHEN SerialNumber like ‘X%’ THEN NULL

WHEN SerialNumber = ‘0’ THEN NULL

WHEN SerialNumber = ‘-‘ THEN NULL

WHEN SerialNumber = ‘Unknown’ THEN NULL

ELSE SerialNumber

END AS SerialNumber_Clean

FROM tblProduct

 

As you can observe, adding an ELSE statement that does not return another explicit NULL makes the script work and as a matter of fact, should be the correct syntax. Interestingly, ELSE is not necessary to make the script valid. The script will run even without an ELSE expression, but only if at least one of the resulting values is not an explicit NULL. The following script runs successfully (notice no ELSE):

 

SELECT ProductID,

CASE WHEN SerialNumber like ‘X%’ THEN NULL

WHEN SerialNumber = ‘0’ THEN NULL

WHEN SerialNumber = ‘-‘ THEN NULL

WHEN SerialNumber = ‘Unknown’  THEN NULL

WHEN SerialNumber = ‘No Serial’ THEN ‘Non-Serialized Product’

END AS SerialNumber_Clean

FROM tblProduct

 

Books on Line should include this caveat on their documentation. http://msdn.microsoft.com/en-us/library/ms181765.aspx

 

The following sections should be modified as follows:

THEN result_expression

Is the expression returned when input_expression equals when_expression evaluates to TRUE, or Boolean_expression evaluates to TRUE. result expression is any valid expression. If no else_result_expression is specified or if else_result_expression is set to return an explicit NULL, at least one result_expression has to be specified to return a non-explicit NULL.

 

ELSE else_result_expression

Is the expression returned if no comparison operation evaluates to TRUE. If this argument is omitted and no comparison operation evaluates to TRUE, CASE returns NULL. else_result_expression is any valid expression. The data types of else_result_expression and any result_expression must be the same or must be an implicit conversion and at least one of them should be specified to return a non-explicit NULL.

GiveCamp Tampa 2011

This past weekend I participated at GiveCamp Tampa 2011 (http://www.givecamptampabay.org/) as part as the Worldwide GiveCamp (http://www.givecamp.org) weekend sponsored by Microsoft and other partners. Per GiveCamp’s website:

GiveCamp is a weekend-long event where technology professionals from designers, developers and database administrators to marketers and web strategists donate their time to provide solutions for non-profit organizations. Since its inception in 2007, the GiveCamp program has provided benefits to hundreds of charities, worth millions of dollars of developer and designer time in services!

At GiveCamp Tampa 2011 we chose two non-profits to be the recipients of our development efforts. David Liebman and a group of developers, implemented a content management site for a school. I architected and developed a Data Warehouse and Business Intelligence solution along with 3 other developers for the Florida Children Services Counsel (FCSC) (http://www.floridacsc.org).

About Florida Children Services Council (FCSC)

The Florida Children’s Services Council (Florida CSC) is a non-profit association that represents children’s services councils in counties throughout the state. Florida CSC employs the collective strengths of these public organizations to improve young lives by making strategic investments in the well being of Florida’s children. Its mission is to promote policies that build effective primary prevention and early intervention systems for young children and their families

The GiveCamp Tampa 2011 Business Intelligence Team
(From left to right: Pam Shaw, Terry Brennan, Paul Drumm, Wes Helton, Jose Chinchilla)

Florida CSC Business Intelligence Solution

The scope of our work included the following tasks:

1. Installation and Configuration of SharePoint 2010 and Performance Point
2. Installation and Configuration of SQL Server 2008R2
3. Architecture and Development of a Relational Data Warehouse
4. Design and Development of ETL framework using SSIS Packages
5. Design and Development of SSAs OLAP cubes
6. Design and Development of SSRS Reports and Performance Point Dashboards

The dataset in scope for the FCSC BI solution included Budget Allocation and Participant Demographic data for the 8 different counties participating in the SAMIS Collaborative including Pinellas, Broward, Martin, Duval, Palm Beach, Miami-Dade, Hillsborough and St. Lucie.

Sample Performance Point dashboard developed:

Afterthoughts

It was overall a great experience being able to dedicate my time to the Florida CSC knowing that the BI solution delivered will serve as starting point to derive insightful analysis to solve the needs of children and families in the State of Florida. More than 90 hours of combined development time was dedicated to this solution. All the team members rocked!

I really enjoyed working with the GiveCamp Tampa 2011 team and look forward to the next GiveCamp event!

Todo lo que debes saber sobre SSIS en 1 hora!

Gracias a lo asistentes de mi charla Todo lo que debes saber sobre SSIS en 1 hora! en el evento 24HOP LATAM. La verdad me agrado mucho presentar en espanol. Aun cuando mi primera lengua es espanol, todas las presentaciones tecnicas en el pasado las he realizado en ingles.

Si deseas ver la presentacion PowerPoint puedes bajarla dando click en la imagen a continuacion.

Tambien puedes bajar el proyecto que utilize en mi presentacion dando click en la imagen a continuacion. Nota: Solo podras utilizar la base de datos y proyecto en SQL Server Denali, no servira en SQL Server 2008 o 2005.

T-SQL: Identifying, inserting and removing spaces in strings

Frequently, when working with strings you will need to identify, insert or remove spaces before, after or in between characters.

For example, you may want to show customer full names  by concatenating the columns that hold the different parts of customer names like FirstName, MiddleName, and LastName, separating each name part with a blank space. You may also want to remove spaces before, after or in between characters from email or website addresses. It is important to understand the different string functions available for you in T-SQL that may help you accomplish any of the tasks mentioned above:

Using single-quotes with space in between  ‘ ‘ versus char(32) to insert spaces between characters

Example #1:

DECLARE @FirstName varchar(25), @LastName varchar(25)
SET @FirstName = ‘Jose’
SET @LastName = ‘Chinchilla’
SELECT @FirstName + ‘ ‘ + @LastName — (one space between single-quotes)
–Result=> Jose Chinchilla

Example #2:

SELECT @FirstName + char(32) + @LastName — (single space special character represented by char(32))
–Result=> Jose Chinchilla

From these two examples we can observe that we get the same results by using single quotes with an empty space in between and char(32). The reason is because char(32) is the ASCII code value of a space character in T-SQL.

SELECT ASCII(‘ ‘)
Result=> 32

SELECT CHAR(ASCII(‘ ‘))
Result=> _  — (the underscore character ‘_’ represents a blank space for clarification purposes, no actual underscores will show)

SELECT CHAR(32)
Result=> _ — (the underscore character ‘_’ represents a blank space for clarification purposes, no actual underscores will show)

Using LEN vs DATALENGTH to get the number of characters in a string

Example #1:

DECLARE @FullName varchar(25)
SET @FullName = ‘ Jose Chinchilla ‘ — (notice spaces before and after the full name)
SELECT LEN(@FullName) –Result=>16

LEN will only count leading spaces and spaces in between the string but will not count trailing spaces.

Example #2:

SELECT DATALENGTH(@FullName)
–Result=>17

DATALENGTH will count spaces in between and both leading and trailing spaces as well.

Using LTRIM and RTRIM to remove leading and trailing spaces

DECLARE @FirstName varchar(25)
SET @FirstName = ‘ Jose ‘; — (notice space before and after the word Jose)

SELECT @FirstName
–Result=> _Jose_  — (the underscore characters ‘_’ represent a blank space for clarification purposes, no actual underscores will show) 

SELECT LTRIM(@FirstName)
–Result=> Jose_  –(only leading space is removed, the underscore character ‘_’ represent a blank space for clarification purposes, no actual underscore will show)

SELECT RTRIM(@FirstName)
–Result=> _Jose  — (only trailing space is removed, the underscore character ‘_’ represent a blank space for clarification purposes, no actual underscore will show)

SELECT LTRIM(RTRIM(@FirstName))
–Result=> Jose  — (both leading and trailing spaces are removed)

By nesting LTRIM and RTRIM you get similar results as the TRIM function in Excel and other programming languages. Unfortunately, there is no TRIM function in T-SQL.

Using CHARINDEX to find the position of all spaces in a string

In order for CHARINDEX to work properly you need to declare the data type length. For example: DECLARE @EmailAddress varchar(100) instread of DECLARE @EmailAddress  varchar. The same applies to nvarchar, char and nchar. You may use varchar(max) and nvarchar(max).

It is important to understand that CHARINDEX will only return the position of the first instance of the character you are looking for, in this case a space or char(32). To continue looking for the rest of the positions where you have empty spaces in your @EmailAddress variable you would need to loop through the entire string until no more spaces are found.

DECLARE @EmailAddress varchar(100), @SpacePositions varchar(max), @PositionIndex int
SET @EmailAddress = ‘ jchinchilla @ sqljoe.com ‘ — (notice leading and trailing spaces and spaces before & after the @ sign)
SET @SpacePositions = ”
SET @PositionIndex = CHARINDEX(char(32),@EmailAddress)

WHILE @PositionIndex > 0
BEGIN
SET @SpacePositions =
CASE
    WHEN CHARINDEX(char(32),@EmailAddress,@PositionIndex+1) > 0
THEN @SpacePositions + CONVERT(varchar,@PositionIndex) + ‘, ‘
ELSE @SpacePositions + CONVERT(varchar,@PositionIndex)
END

SET @PositionIndex = CHARINDEX(char(32),@EmailAddress,@PositionIndex+1)
END

SELECT @SpacePositions as  SpacePositions
–Result=> 1, 13, 15, 26

Using REPLACE to find and remove all spaces in a string

REPLACE will allow you to substitute all spaces in a string with an empty string, thus removing all spaces in between characters. In the following example, all spaces represented by char(32) will be replaced with an empty string ”.

DECLARE @EmailAddress varchar(max)
SET @EmailAddress = ‘ jchinchilla @ sqljoe.com ‘ — (notice leading and traling spaces and spaces before and after the @ sign)

SELECT REPLACE(@EmailAddress, CHAR(32),”)
–Result=> jchinchilla@sqljoe.com –(all spaces have been removed)

Using REPLICATE to insert ‘n’ number of spaces in a string

SELECT  ‘A’ + REPLICATE(‘ ‘,5)  + ‘Z’
Result=> A_____Z  — (the underscore characters ‘_’ represent a blank space for clarification purposes, no actual underscores will show)

SELECT  ‘A’ + REPLICATE(char(32),5)  + ‘Z’
Result=> A_____Z  — (the underscore characters ‘_’ represent a blank space for clarification purposes, no actual underscores will show)

SSIS Foreach File Enumerator returns more files than expected by appending a wildcard (*) to the file mask

While working on a recent project for a customer that involved importing both Excel 97-2003 and Excel 2007/2010 files, I was a little surprised to discover that the Foreach Loop File Enumerator will return both *.xls and *.xlsx files even if you only specify to return *.xls files.

I tested this behaviour with similar results with other file formats as well. During my testing, I created three files:

1. File_txt.txt
2. File_txts.txts
3. File_txtsy.txtsy

See the screenshot of the three test files created below:

Next, I created an SSIS Package with a Foreach Loop Container with Foreach File Enumerator as the enumerator type and  specified *.txt as the file mask. The file names read by the Foreach Loop Container are being assigned to a user variable called varFileName. Then, inside the Foreach Loop Container I added a simple VB Script Task that returns the value of the user variable varFileName inside a message box.

See the Foreach Loop Containter configurations below:

See the VB script inside the Script Task below: 

You would expect only file_txt.txt to be returned, but as it turns out all three files were returned as can bee seen on the screen captures for the message boxes below:

It looks like the Foreach File Enumerator appends an asterisk (*) at the end of the extension portion of the file mask you specify. This means, that specifying *.txt or *.xls is the same as specifying *.txt* or *.xls*. As an additional test, I executed the <dir> command in D.O.S. to see the results of both masks. The two commands executed are shown below along with the results:

Interestingly, we get the same result in D.O.S. if we specify *.txt or *.txt*. I assume then, that the Foreach Loop file enumeration behaviour may be bound to the D.O.S. output of the Operating System. The current version of D.O.S. on my Windows 7 machine is 6.1.7600. With the introduction of support for more than three letter extensions, this little issue might have been overlooked in D.O.S. for the <dir> command.

The only reference I found regarding this issue with SSIS and the For Each Loop Container was by Douglas Laudenschlager (Blog), technical writer for Microsoft on the SQL Server Integration Services documentation team. The blog post where he mentions this as a gotcha can be found here: http://dougbert.com/blogs/dougbert/archive/2008/06/16/excel-in-integration-services-part-1-of-3-connections-and-components.aspx

Douglas correctly expresses,

 There appears to be no way to specify, “Give me .xls but not .xlsx”.

So, if you require to only limit your control flow execution for files with a specific extension, as in Douglas’ example .xls but not .xlsx, then your alternative will be to assign to a separate variable the extension portion of the file name contained in the varFileName variable of my SSIS package above. Once you capture the file extension in a variable, you can use a precedence constraint to restrict further control fow task execution for files of a specific file extension.

In my case, I declared a variable called varFileExtension and added the following line to the existing Script Task in my example:



I can now use a precedence constraint to check if the value contained in my varFileExtension is equal to “txt” as follows:

 By specifying a precedence constraint that evaluates for the exact file extension I’m looking to work with, I can limit the rest of the execution for only these files. Furthermore, if I wanted to have separate control flow logics for files of type “.xls” and “.xlsx” for example, I can split my control flow using two precedence constraints. In such case I could change the code inside my script to include the last 4 characters instead of just three. My two precedence constraint expressions would look something like this then:

1. For XLS files -> @[User::varFileExtension]==”.xls” (period included since we are assigning the last 4 characters now)
2. For XLSX files -> @[User::varFileExtension]==”xlsx”

The SSIS package would look something like this now:

 

Differences between Merge and Union All transformations in SSIS

 What are the differences between Merge and Union All transformations in SSIS ?

 The first and most obvious difference is that Merge can only accept two datasets while Union All can accept more than two datasets for input. The second difference is that Merge requires both datasets to be sorted while Union All does not require sorted datasets. Both transformations are considered to be partially-blocking.

As can be seen on the following image, Union All can accept more than two datasets as input while Merge is limited to only two input datasets. Also, notice that for the Merge transformation, the datasets need to be sorted before they can be accepted as valid input.

For simplification purposes,  a Sort transformation was used. Keep in mind that the Sort transformation is a full blocking operation because it needs to read all the rows in the data flow (buffer) in order to sort the data. No data will be sent downstream until all rows have been read first by the Sort transformation. A better approach would be to sort the datasets with an ORDER BY T-SQL clause at the data source component.