Showing posts with label scan. Show all posts
Showing posts with label scan. Show all posts

Wednesday, March 28, 2012

Optimizing Data Warehouse Query Performance Through Bitmap Filtering...

Hi, as per the BOL, In the "Optimizing Data Warehouse Query Performance Through Bitmap Filtering"

the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai according to sql Server 2008.

Can you verify it please.

Hi Akash,

What statements in the topic led you to that conclusion? I wrote the topic, so I want to make sure I correct any such sentences that can be perceived incorrectly.

Thanks,

Gail

|||

Hi Akash,

technically, there is no reason for higher cost of scans. Could you please point Gail to the BOL topic so that we can make sure that it is accurate?

Thanks,

Torsten Grabs [MSFT]

Program Manager

SQL Server Query Processor

|||

Hi Gail,

As i've mentioned that in the BOL when you type "Bitmap Index", you let the user understand how optimized bitmap filtering is implemented. You also demonstrate an example for it comparing both the SQL Server 2005 along with Katmai and in that very example you did mention that the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai when compared to sql Server 2005. Just have a look at the figures.

Clustered index scan costs 3% for DimProduct & 12% for DimCustomer in case of SQL Server 2008 where as the same clustering index scan costs 2% and 11% respectively in SQL Server 2005.

Similarly Table Scan costs 14% in SQL Server 2008 and 13% in case of SQL Server 2005.

Should not there be a decrese in scanning cost in Katmai when compared to SQL Server 2005 ?

|||

Hi Akash,

Thanks very much for the pointer to the BOL entry you were referring to. There are two considerations two keep in mind when comparing the table scan cost information as you do:

- Costing functions may change between releases. Hence, when you compare query plan costs and operator costs across releases, you are on unstable ground as you make the assumption that the costing function has not been changed.

- The percentages are relative costs. The less expensive the remaining query plan, i.e., the plan on top of the table scan, gets the higher this percentage value will be - assuming that the cost of the scan is the same. If you look into the subtree cost information of the scan, you will see that the cost is the same for both plans. But, the remaining operators, in particular the hash joins on top, have become cheaper since there are fewer rows to process - which explain why you see the percentages increase.

A further note on why the absolute costs between the scan operators remain the same: even with bitmap filters, the table scan operation has to read all the rows from the disk or buffer pool initially. However, as soon as the row is read into the query processor, we can apply the bitmap filter. The main cost, however, is spent in disk or buffer pool reads - the bitmap filter processing itself is very cheap. Therefore, we do not model the bitmap filtering cost in plan generation. However, optimized bitmap filters do model the cardinality changes of the bitmap filter on top of the scan operation.

Hope this explains the showplan information in the BOL topic.

Best regards,

Torsten Grabs [MSFT]

Program Manager

Microsoft SQL Server Query Processor

|||Thanks Torsten. It was really a great help,appreciate your effort in letting me understand this

Optimizing Data Warehouse Query Performance Through Bitmap Filtering...

Hi, as per the BOL, In the "Optimizing Data Warehouse Query Performance Through Bitmap Filtering"

the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai according to sql Server 2008.

Can you verify it please.

Hi Akash,

What statements in the topic led you to that conclusion? I wrote the topic, so I want to make sure I correct any such sentences that can be perceived incorrectly.

Thanks,

Gail

|||

Hi Akash,

technically, there is no reason for higher cost of scans. Could you please point Gail to the BOL topic so that we can make sure that it is accurate?

Thanks,

Torsten Grabs [MSFT]

Program Manager

SQL Server Query Processor

|||

Hi Gail,

As i've mentioned that in the BOL when you type "Bitmap Index", you let the user understand how optimized bitmap filtering is implemented. You also demonstrate an example for it comparing both the SQL Server 2005 along with Katmai and in that very example you did mention that the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai when compared to sql Server 2005. Just have a look at the figures.

Clustered index scan costs 3% for DimProduct & 12% for DimCustomer in case of SQL Server 2008 where as the same clustering index scan costs 2% and 11% respectively in SQL Server 2005.

Similarly Table Scan costs 14% in SQL Server 2008 and 13% in case of SQL Server 2005.

Should not there be a decrese in scanning cost in Katmai when compared to SQL Server 2005 ?

|||

Hi Akash,

Thanks very much for the pointer to the BOL entry you were referring to. There are two considerations two keep in mind when comparing the table scan cost information as you do:

- Costing functions may change between releases. Hence, when you compare query plan costs and operator costs across releases, you are on unstable ground as you make the assumption that the costing function has not been changed.

- The percentages are relative costs. The less expensive the remaining query plan, i.e., the plan on top of the table scan, gets the higher this percentage value will be - assuming that the cost of the scan is the same. If you look into the subtree cost information of the scan, you will see that the cost is the same for both plans. But, the remaining operators, in particular the hash joins on top, have become cheaper since there are fewer rows to process - which explain why you see the percentages increase.

A further note on why the absolute costs between the scan operators remain the same: even with bitmap filters, the table scan operation has to read all the rows from the disk or buffer pool initially. However, as soon as the row is read into the query processor, we can apply the bitmap filter. The main cost, however, is spent in disk or buffer pool reads - the bitmap filter processing itself is very cheap. Therefore, we do not model the bitmap filtering cost in plan generation. However, optimized bitmap filters do model the cardinality changes of the bitmap filter on top of the scan operation.

Hope this explains the showplan information in the BOL topic.

Best regards,

Torsten Grabs [MSFT]

Program Manager

Microsoft SQL Server Query Processor

|||Thanks Torsten. It was really a great help,appreciate your effort in letting me understand this

Optimizing Data Warehouse Query Performance Through Bitmap Filtering...

Hi, as per the BOL, In the "Optimizing Data Warehouse Query Performance Through Bitmap Filtering"

the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai according to sql Server 2008.

Can you verify it please.

Hi Akash,

What statements in the topic led you to that conclusion? I wrote the topic, so I want to make sure I correct any such sentences that can be perceived incorrectly.

Thanks,

Gail

|||

Hi Akash,

technically, there is no reason for higher cost of scans. Could you please point Gail to the BOL topic so that we can make sure that it is accurate?

Thanks,

Torsten Grabs [MSFT]

Program Manager

SQL Server Query Processor

|||

Hi Gail,

As i've mentioned that in the BOL when you type "Bitmap Index", you let the user understand how optimized bitmap filtering is implemented. You also demonstrate an example for it comparing both the SQL Server 2005 along with Katmai and in that very example you did mention that the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai when compared to sql Server 2005. Just have a look at the figures.

Clustered index scan costs 3% for DimProduct & 12% for DimCustomer in case of SQL Server 2008 where as the same clustering index scan costs 2% and 11% respectively in SQL Server 2005.

Similarly Table Scan costs 14% in SQL Server 2008 and 13% in case of SQL Server 2005.

Should not there be a decrese in scanning cost in Katmai when compared to SQL Server 2005 ?

|||

Hi Akash,

Thanks very much for the pointer to the BOL entry you were referring to. There are two considerations two keep in mind when comparing the table scan cost information as you do:

- Costing functions may change between releases. Hence, when you compare query plan costs and operator costs across releases, you are on unstable ground as you make the assumption that the costing function has not been changed.

- The percentages are relative costs. The less expensive the remaining query plan, i.e., the plan on top of the table scan, gets the higher this percentage value will be - assuming that the cost of the scan is the same. If you look into the subtree cost information of the scan, you will see that the cost is the same for both plans. But, the remaining operators, in particular the hash joins on top, have become cheaper since there are fewer rows to process - which explain why you see the percentages increase.

A further note on why the absolute costs between the scan operators remain the same: even with bitmap filters, the table scan operation has to read all the rows from the disk or buffer pool initially. However, as soon as the row is read into the query processor, we can apply the bitmap filter. The main cost, however, is spent in disk or buffer pool reads - the bitmap filter processing itself is very cheap. Therefore, we do not model the bitmap filtering cost in plan generation. However, optimized bitmap filters do model the cardinality changes of the bitmap filter on top of the scan operation.

Hope this explains the showplan information in the BOL topic.

Best regards,

Torsten Grabs [MSFT]

Program Manager

Microsoft SQL Server Query Processor

|||

Thanks Torsten. It was really a great help,appreciate your effort in letting me understand this

Optimizing Data Warehouse Query Performance Through Bitmap Filtering...

Hi, as per the BOL, In the "Optimizing Data Warehouse Query Performance Through Bitmap Filtering"

the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai according to sql Server 2008.

Can you verify it please.

Hi Akash,

What statements in the topic led you to that conclusion? I wrote the topic, so I want to make sure I correct any such sentences that can be perceived incorrectly.

Thanks,

Gail

|||

Hi Akash,

technically, there is no reason for higher cost of scans. Could you please point Gail to the BOL topic so that we can make sure that it is accurate?

Thanks,

Torsten Grabs [MSFT]

Program Manager

SQL Server Query Processor

|||

Hi Gail,

As i've mentioned that in the BOL when you type "Bitmap Index", you let the user understand how optimized bitmap filtering is implemented. You also demonstrate an example for it comparing both the SQL Server 2005 along with Katmai and in that very example you did mention that the cost of Clustered Index Scan as well as Table Scan takes more time in Katmai when compared to sql Server 2005. Just have a look at the figures.

Clustered index scan costs 3% for DimProduct & 12% for DimCustomer in case of SQL Server 2008 where as the same clustering index scan costs 2% and 11% respectively in SQL Server 2005.

Similarly Table Scan costs 14% in SQL Server 2008 and 13% in case of SQL Server 2005.

Should not there be a decrese in scanning cost in Katmai when compared to SQL Server 2005 ?

|||

Hi Akash,

Thanks very much for the pointer to the BOL entry you were referring to. There are two considerations two keep in mind when comparing the table scan cost information as you do:

- Costing functions may change between releases. Hence, when you compare query plan costs and operator costs across releases, you are on unstable ground as you make the assumption that the costing function has not been changed.

- The percentages are relative costs. The less expensive the remaining query plan, i.e., the plan on top of the table scan, gets the higher this percentage value will be - assuming that the cost of the scan is the same. If you look into the subtree cost information of the scan, you will see that the cost is the same for both plans. But, the remaining operators, in particular the hash joins on top, have become cheaper since there are fewer rows to process - which explain why you see the percentages increase.

A further note on why the absolute costs between the scan operators remain the same: even with bitmap filters, the table scan operation has to read all the rows from the disk or buffer pool initially. However, as soon as the row is read into the query processor, we can apply the bitmap filter. The main cost, however, is spent in disk or buffer pool reads - the bitmap filter processing itself is very cheap. Therefore, we do not model the bitmap filtering cost in plan generation. However, optimized bitmap filters do model the cardinality changes of the bitmap filter on top of the scan operation.

Hope this explains the showplan information in the BOL topic.

Best regards,

Torsten Grabs [MSFT]

Program Manager

Microsoft SQL Server Query Processor

|||Thanks Torsten. It was really a great help,appreciate your effort in letting me understand this sql

Optimizing Clustered Index Scan

I am trying to tune a query that has as its execution output as a Clustered
Index Scan. There is a Clustered Index on field [dt], which is a datetim
e
field. The query is structured as such:
IF @.date < '1/1/1901'
--...then make it NULL, so that it works in the query
SET @.date = NULL
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE ( ( @.date IS NULL )
OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
Clustered Index Seek. Is there any way to rework the WHERE clause to where i
t
would always be a seek?
Message posted via droptable.com
http://www.droptable.com/Uwe/Forum...server/200604/1Why are you setting @.date to null if @.date is < '1/1/1901'?
MG
"cbrichards" wrote:

> I am trying to tune a query that has as its execution output as a Clustere
d
> Index Scan. There is a Clustered Index on field [dt], which is a datet
ime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where
it
> would always be a seek?
> --
> Message posted via droptable.com
> http://www.droptable.com/Uwe/Forum...server/200604/1
>|||Try,
IF @.date < '1/1/1901'
--...then make it NULL, so that it works in the query
SET @.date = NULL
SELECT
t1.dt_UID
FROM
Table1 AS t1
WHERE
t1.dt BETWEEN isnull(@.date, '17530101') AND isnull(DateAdd(d, 1, @.date),
'9999-12-30T23:59:59')
go
Dynamic Search Conditions in T-SQL
http://www.sommarskog.se/dyn-search.html
AMB
"cbrichards" wrote:

> I am trying to tune a query that has as its execution output as a Clustere
d
> Index Scan. There is a Clustered Index on field [dt], which is a datet
ime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where
it
> would always be a seek?
> --
> Message posted via droptable.com
> http://www.droptable.com/Uwe/Forum...server/200604/1
>|||"cbrichards" <u3288@.uwe> wrote in message news:5f760d81fa01c@.uwe...
>I am trying to tune a query that has as its execution output as a Clustered
> Index Scan. There is a Clustered Index on field [dt], which is a datet
ime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where
> it
> would always be a seek?
>
This is a classic case of pseudo-dynamic SQL. THis is really two different
queries crammed into one. With this query you must do a scan because the
same plan is used whether or not @.date is null. If it is, obviously only a
clustered index scan will do.
IF @.date < '1/1/1901'
BEGIN
SELECT
t1.dt_UID
FROM Table1 AS t1
END
ELSE
BEGIN
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date)
END
david|||David,
Thanks for your response. I ran this in test and (true enough!) the same pla
n
is used whether or not @.date is null. At this point, I am trying to
understand why the same plan is used. Could you explain further, please,
specifically why the same plan is used?
David Browne wrote:
>[quoted text clipped - 14 lines]
>This is a classic case of pseudo-dynamic SQL. THis is really two different
>queries crammed into one. With this query you must do a scan because the
>same plan is used whether or not @.date is null. If it is, obviously only a
>clustered index scan will do.
>IF @.date < '1/1/1901'
>BEGIN
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
>END
>ELSE
>BEGIN
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date)
>END
>david
Message posted via http://www.droptable.com|||"cbrichards via droptable.com" <u3288@.uwe> wrote in message
news:5f9b39b4a6681@.uwe...
> David,
> Thanks for your response. I ran this in test and (true enough!) the same
> plan
> is used whether or not @.date is null. At this point, I am trying to
> understand why the same plan is used. Could you explain further, please,
> specifically why the same plan is used?
>
Each query gets only one plan. The values you happen to bind to the
parameters don't change that. So whatever the plan is for the query, it has
to work for all possible values of the parameters.
Read:
Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
http://www.microsoft.com/technet/pr...005/recomp.mspx
David|||Thanks Alejandro. That appears to resolve the scan regardless of the value o
f
@.date. Much appreciated. Thanks for the link too.
Alejandro Mesa wrote:[vbcol=seagreen]
>Try,
>IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
>SELECT
> t1.dt_UID
>FROM
> Table1 AS t1
>WHERE
> t1.dt BETWEEN isnull(@.date, '17530101') AND isnull(DateAdd(d, 1, @.date),
>'9999-12-30T23:59:59')
>go
>Dynamic Search Conditions in T-SQL
>http://www.sommarskog.se/dyn-search.html
>AMB
>
>[quoted text clipped - 13 lines]
Message posted via http://www.droptable.com|||David,
I read through the article you provided as a link. I understand better what
you mean by "each query gets only one plan" and "parameters do not change
that."
I am trying to internalize this knowledge and understand it within the
context of my attached statement, which contains the OR condition. As you
originally mentioned, I really have "two different queries crammed into one.
"
As I originally noted, when I remove the ( @.date IS NULL ) portion from the
WHERE clause I get a Clustered Index Seek. Does this mean, that when the pla
n
is created, that in order for the optimizer to cache only one plan (in this
case, using my pseudo dynamic SQL), the optimizer, in essence, will go with
a
plan that will satisfy both conditions, and the plan that satisfies both
conditions is an Index Scan? Is my understanding correct?
David Browne wrote:
>Each query gets only one plan. The values you happen to bind to the
>parameters don't change that. So whatever the plan is for the query, it ha
s
>to work for all possible values of the parameters.
>Read:
>Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 200
5
>http://www.microsoft.com/technet/pr...005/recomp.mspx
>David
Message posted via droptable.com
http://www.droptable.com/Uwe/Forum...server/200605/1|||"cbrichards via droptable.com" <u3288@.uwe> wrote in message
news:5f9e9495ccc86@.uwe...
> David,
> I read through the article you provided as a link. I understand better
> what
> you mean by "each query gets only one plan" and "parameters do not change
> that."
> I am trying to internalize this knowledge and understand it within the
> context of my attached statement, which contains the OR condition. As you
> originally mentioned, I really have "two different queries crammed into
> one."
> As I originally noted, when I remove the ( @.date IS NULL ) portion from
> the
> WHERE clause I get a Clustered Index Seek. Does this mean, that when the
> plan
> is created, that in order for the optimizer to cache only one plan (in
> this
> case, using my pseudo dynamic SQL), the optimizer, in essence, will go
> with a
> plan that will satisfy both conditions, and the plan that satisfies both
> conditions is an Index Scan? Is my understanding correct?
>
Yes. That is it, exactly.
David|||you would have a LOT better luck if you added anohter variable, and did
the dateadd before the select statement.
The way you wrote it, the system has to calculate the dateadd for each
and every row, and it might not be obvious to the engine how simple
your querty could be.
set @.date2 =DateAdd(d, 1, @.date)
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE ( t1.dt BETWEEN @.date AND @.date2)
OR ( @.date IS NULL )
sounds weird, but sometimes moving the null part around encourages or
discourages the use of hte index. Depends on how that particular engine
parses - you want it to parse and sue the index first.
Worst case you can use a hint to force the engine to use the index.

Optimizing Clustered Index Scan

I am trying to tune a query that has as its execution output as a Clustered
Index Scan. There is a Clustered Index on field [dt], which is a datetime
field. The query is structured as such:
IF @.date < '1/1/1901'
--...then make it NULL, so that it works in the query
SET @.date = NULL
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE ( ( @.date IS NULL )
OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
Clustered Index Seek. Is there any way to rework the WHERE clause to where it
would always be a seek?
--
Message posted via SQLMonster.com
http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200604/1Why are you setting @.date to null if @.date is < '1/1/1901'?
--
MG
"cbrichards" wrote:
> I am trying to tune a query that has as its execution output as a Clustered
> Index Scan. There is a Clustered Index on field [dt], which is a datetime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where it
> would always be a seek?
> --
> Message posted via SQLMonster.com
> http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200604/1
>|||Try,
IF @.date < '1/1/1901'
--...then make it NULL, so that it works in the query
SET @.date = NULL
SELECT
t1.dt_UID
FROM
Table1 AS t1
WHERE
t1.dt BETWEEN isnull(@.date, '17530101') AND isnull(DateAdd(d, 1, @.date),
'9999-12-30T23:59:59')
go
Dynamic Search Conditions in T-SQL
http://www.sommarskog.se/dyn-search.html
AMB
"cbrichards" wrote:
> I am trying to tune a query that has as its execution output as a Clustered
> Index Scan. There is a Clustered Index on field [dt], which is a datetime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where it
> would always be a seek?
> --
> Message posted via SQLMonster.com
> http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200604/1
>|||"cbrichards" <u3288@.uwe> wrote in message news:5f760d81fa01c@.uwe...
>I am trying to tune a query that has as its execution output as a Clustered
> Index Scan. There is a Clustered Index on field [dt], which is a datetime
> field. The query is structured as such:
>
> IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE ( ( @.date IS NULL )
> OR ( t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date) ) )
> When I remove the ( @.date IS NULL ) portion from the WHERE clause I get a
> Clustered Index Seek. Is there any way to rework the WHERE clause to where
> it
> would always be a seek?
>
This is a classic case of pseudo-dynamic SQL. THis is really two different
queries crammed into one. With this query you must do a scan because the
same plan is used whether or not @.date is null. If it is, obviously only a
clustered index scan will do.
IF @.date < '1/1/1901'
BEGIN
SELECT
t1.dt_UID
FROM Table1 AS t1
END
ELSE
BEGIN
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date)
END
david|||David,
Thanks for your response. I ran this in test and (true enough!) the same plan
is used whether or not @.date is null. At this point, I am trying to
understand why the same plan is used. Could you explain further, please,
specifically why the same plan is used?
David Browne wrote:
>>I am trying to tune a query that has as its execution output as a Clustered
>> Index Scan. There is a Clustered Index on field [dt], which is a datetime
>[quoted text clipped - 14 lines]
>> it
>> would always be a seek?
>This is a classic case of pseudo-dynamic SQL. THis is really two different
>queries crammed into one. With this query you must do a scan because the
>same plan is used whether or not @.date is null. If it is, obviously only a
>clustered index scan will do.
>IF @.date < '1/1/1901'
>BEGIN
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
>END
>ELSE
>BEGIN
> SELECT
> t1.dt_UID
> FROM Table1 AS t1
> WHERE t1.dt BETWEEN @.date AND DateAdd(d, 1, @.date)
>END
>david
--
Message posted via http://www.sqlmonster.com|||"cbrichards via SQLMonster.com" <u3288@.uwe> wrote in message
news:5f9b39b4a6681@.uwe...
> David,
> Thanks for your response. I ran this in test and (true enough!) the same
> plan
> is used whether or not @.date is null. At this point, I am trying to
> understand why the same plan is used. Could you explain further, please,
> specifically why the same plan is used?
>
Each query gets only one plan. The values you happen to bind to the
parameters don't change that. So whatever the plan is for the query, it has
to work for all possible values of the parameters.
Read:
Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
http://www.microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx
David|||Thanks Alejandro. That appears to resolve the scan regardless of the value of
@.date. Much appreciated. Thanks for the link too.
Alejandro Mesa wrote:
>Try,
>IF @.date < '1/1/1901'
> --...then make it NULL, so that it works in the query
> SET @.date = NULL
>SELECT
> t1.dt_UID
>FROM
> Table1 AS t1
>WHERE
> t1.dt BETWEEN isnull(@.date, '17530101') AND isnull(DateAdd(d, 1, @.date),
>'9999-12-30T23:59:59')
>go
>Dynamic Search Conditions in T-SQL
>http://www.sommarskog.se/dyn-search.html
>AMB
>> I am trying to tune a query that has as its execution output as a Clustered
>> Index Scan. There is a Clustered Index on field [dt], which is a datetime
>[quoted text clipped - 13 lines]
>> Clustered Index Seek. Is there any way to rework the WHERE clause to where it
>> would always be a seek?
--
Message posted via http://www.sqlmonster.com|||David,
I read through the article you provided as a link. I understand better what
you mean by "each query gets only one plan" and "parameters do not change
that."
I am trying to internalize this knowledge and understand it within the
context of my attached statement, which contains the OR condition. As you
originally mentioned, I really have "two different queries crammed into one."
As I originally noted, when I remove the ( @.date IS NULL ) portion from the
WHERE clause I get a Clustered Index Seek. Does this mean, that when the plan
is created, that in order for the optimizer to cache only one plan (in this
case, using my pseudo dynamic SQL), the optimizer, in essence, will go with a
plan that will satisfy both conditions, and the plan that satisfies both
conditions is an Index Scan? Is my understanding correct?
David Browne wrote:
>> David,
>> Thanks for your response. I ran this in test and (true enough!) the same
>> plan
>> is used whether or not @.date is null. At this point, I am trying to
>> understand why the same plan is used. Could you explain further, please,
>> specifically why the same plan is used?
>Each query gets only one plan. The values you happen to bind to the
>parameters don't change that. So whatever the plan is for the query, it has
>to work for all possible values of the parameters.
>Read:
>Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
>http://www.microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx
>David
--
Message posted via SQLMonster.com
http://www.sqlmonster.com/Uwe/Forums.aspx/sql-server/200605/1|||"cbrichards via SQLMonster.com" <u3288@.uwe> wrote in message
news:5f9e9495ccc86@.uwe...
> David,
> I read through the article you provided as a link. I understand better
> what
> you mean by "each query gets only one plan" and "parameters do not change
> that."
> I am trying to internalize this knowledge and understand it within the
> context of my attached statement, which contains the OR condition. As you
> originally mentioned, I really have "two different queries crammed into
> one."
> As I originally noted, when I remove the ( @.date IS NULL ) portion from
> the
> WHERE clause I get a Clustered Index Seek. Does this mean, that when the
> plan
> is created, that in order for the optimizer to cache only one plan (in
> this
> case, using my pseudo dynamic SQL), the optimizer, in essence, will go
> with a
> plan that will satisfy both conditions, and the plan that satisfies both
> conditions is an Index Scan? Is my understanding correct?
>
Yes. That is it, exactly.
David|||you would have a LOT better luck if you added anohter variable, and did
the dateadd before the select statement.
The way you wrote it, the system has to calculate the dateadd for each
and every row, and it might not be obvious to the engine how simple
your querty could be.
set @.date2 =DateAdd(d, 1, @.date)
SELECT
t1.dt_UID
FROM Table1 AS t1
WHERE ( t1.dt BETWEEN @.date AND @.date2)
OR ( @.date IS NULL )
sounds weird, but sometimes moving the null part around encourages or
discourages the use of hte index. Depends on how that particular engine
parses - you want it to parse and sue the index first.
Worst case you can use a hint to force the engine to use the index.

Optimizing a query

Hi,
I have created the following SP and indexes, but the execution plan for SP
shows that query optimizer always uses 'index scan'.
--
use northwind
go
create proc usp_search
@.country varchar(100)=null,
@.city varchar(100)=null
as
select customerid,companyname,country,city from customers where
(@.country is null OR country=@.country)
and
(@.city is null OR city=@.city)
go
create index ix10 on customers(country,city,companyname)
create index ix11 on customers(city,country,companyname)
go
exec usp_search 'uk','london' with recompile
--
If I remove any of the ORs, one of my indxes will be used. Are there any
solution to keep both ORs and optimizer uses my indexes? Should I force
optimizer to use any index?
Thanks in advance,
LeilaLeila,
in this case it helps to be specific. Since you only have 4 cases, a
nested IF ... ELSE will do the trick
*untested*:
if (@.country is null)
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
end
end
else
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
where country=@.country
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
and country=@.country
end
end
Good luck!|||You might want to start from:
http://www.sommarskog.se/dyn-search.html
Anith|||Thanks Alexander,
But my real SP has 20 parameters. This SP was only a sample of what I want
to do.
"Alexander Kuznetsov" <AK_TIREDOFSPAM@.hotmail.COM> wrote in message
news:1138634724.575101.70310@.g44g2000cwa.googlegroups.com...
> Leila,
> in this case it helps to be specific. Since you only have 4 cases, a
> nested IF ... ELSE will do the trick
> *untested*:
> if (@.country is null)
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> end
> end
> else
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> where country=@.country
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> and country=@.country
> end
> end
> Good luck!
>|||Leila,
In that case I would concur with Anith. I would utilize dynamic SQL, as
it is described in Erland's article he mentioned.
Yet I have a question for you. How are you testing your SP with 20
parameters? With 8 ro 10 parameters I would do something like this:
create table test_log(country varchar(25), city varchar(25))
go
create procedure myproc(@.country varchar(25), @.city varchar(25))
as
insert into test_log values(@.country, @.city)
go
declare @.country varchar(25), @.city varchar(25)
declare test_cases cursor
for
select * from
(
-- more than 50% customers
select 'USA' country_name
union all
-- less then 1% customers
select 'New Zealand'
union all
select NULL) country,
(select 'Boston' city
union all
-- city inconsistent with any country from the list above
select 'Kharkiv' city
union all
select null
) city
open test_cases
fetch next from test_cases into @.country, @.city
while @.@.fetch_status=0
begin
exec myproc @.country, @.city
fetch next from test_cases into @.country, @.city
end
go
select * from test_log
country city
-- --
USA Boston
USA Kharkiv
USA NULL
New Zealand Boston
New Zealand Kharkiv
New Zealand NULL
NULL Boston
NULL Kharkiv
NULL NULL
(9 row(s) affected)
go
drop table test_log
drop procedure myproc
So, for 2 parameters I needed 9 calls to do a unit test. Of course,
there is no need to open a cursor for mere 9 calls, I just wanted to
demostrate the technique used to make 1K calls.
Are you making 1 million calls for unit testing of your procedure with
20 parameters?|||This is a classic example where dynamic execution should be considered.
You can find details here, assuming you have a subscription to SQLMag:
http://www.windowsitpro.com/Article...7502/47502.html
If you don't, let me know and I'll try to summarize.
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
> Hi,
> I have created the following SP and indexes, but the execution plan for SP
> shows that query optimizer always uses 'index scan'.
> --
> use northwind
> go
> create proc usp_search
> @.country varchar(100)=null,
> @.city varchar(100)=null
> as
> select customerid,companyname,country,city from customers where
> (@.country is null OR country=@.country)
> and
> (@.city is null OR city=@.city)
> go
> create index ix10 on customers(country,city,companyname)
> create index ix11 on customers(city,country,companyname)
> go
> exec usp_search 'uk','london' with recompile
> --
>
> If I remove any of the ORs, one of my indxes will be used. Are there any
> solution to keep both ORs and optimizer uses my indexes? Should I force
> optimizer to use any index?
> Thanks in advance,
> Leila
>
>|||Thanks Itzik,
I'll be most grateful if you could do that.
BTW, what's your idea about this manner:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
It works very good and performs an 'Index S'. But one of the problems
that I noticed is on numeric columns(parameters). It has to implicitly
convert the number to varchar, so it doesn't perform Index s, rather it
does Index Scan. I mean in the worst situation, its performance is like the
SP which I wrote in my first post (using NULLs)
Leila
"Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in message
news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
> This is a classic example where dynamic execution should be considered.
> You can find details here, assuming you have a subscription to SQLMag:
> http://www.windowsitpro.com/Article...7502/47502.html
> If you don't, let me know and I'll try to summarize.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>
> "Leila" <Leilas@.hotpop.com> wrote in message
> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>|||If you had followed any ISO standards instead of what you made up on
the fly, would it look more like this? Without the super long
parameters that invite errors? With an ISO-11179 names?
CREATE PROCEDURE SearchCity
(@.my_country_code CHAR(3) = NULL, -- ISO standards!!
@.my_city_naem VARCHAR(25) = NULL -- postal union standards
AS
SELECT customer_id, company_name, country_code, city_name
FROM Customers
WHERE COALESCE (@.my_country_code, country_code = country_code)
AND COALESCE (@.my_ city_name, city_name) = city_anme ;
Since you never thought to post DDL, can we assume that (company_name,
city_name, country_code) is the key? The usual rule is to order an
index by the most selective to the least selective column.
SQL Server's optimizer is still a bit behind, so it the COALESCE()
trick does not work as well as it does in other products, such as DB2,
that can spot this form. I am not sure if SQL-2005 can do it.|||Thanks Joe,
The COALESCE function (in SQL Server 2005) produces the same execution plan
as using IS NULL manner (an index scan is performed). But using '%' and
'like' performs index s when you use it to seach strings:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>|||Again you show your complete lack of real world implementation experience of
SQL.
The query you present will give a table or index scan and will not scale, if
will cause SIGNIFICANT performance problems on a large table.
You should use IF ELSE at the very least to code for each optional parameter
combination, this can be done in the stored procedure, a single stored
procedure without having to bloat code and go for multiple stored procedures
which would lead to a more complicated design and increase your development
and maintanence costs.
Tony Rogerson
SQL Server MVP
http://sqlserverfaq.com - free video tutorials
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>

Optimizing a query

Hi,
I have created the following SP and indexes, but the execution plan for SP
shows that query optimizer always uses 'index scan'.
use northwind
go
create proc usp_search
@.country varchar(100)=null,
@.city varchar(100)=null
as
select customerid,companyname,country,city from customers where
(@.country is null OR country=@.country)
and
(@.city is null OR city=@.city)
go
create index ix10 on customers(country,city,companyname)
create index ix11 on customers(city,country,companyname)
go
exec usp_search 'uk','london' with recompile
If I remove any of the ORs, one of my indxes will be used. Are there any
solution to keep both ORs and optimizer uses my indexes? Should I force
optimizer to use any index?
Thanks in advance,
Leila
Leila,
in this case it helps to be specific. Since you only have 4 cases, a
nested IF ... ELSE will do the trick
*untested*:
if (@.country is null)
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
end
end
else
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
where country=@.country
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
and country=@.country
end
end
Good luck!
|||You might want to start from:
http://www.sommarskog.se/dyn-search.html
Anith
|||Thanks Alexander,
But my real SP has 20 parameters. This SP was only a sample of what I want
to do.
"Alexander Kuznetsov" <AK_TIREDOFSPAM@.hotmail.COM> wrote in message
news:1138634724.575101.70310@.g44g2000cwa.googlegro ups.com...
> Leila,
> in this case it helps to be specific. Since you only have 4 cases, a
> nested IF ... ELSE will do the trick
> *untested*:
> if (@.country is null)
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> end
> end
> else
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> where country=@.country
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> and country=@.country
> end
> end
> Good luck!
>
|||Leila,
In that case I would concur with Anith. I would utilize dynamic SQL, as
it is described in Erland's article he mentioned.
Yet I have a question for you. How are you testing your SP with 20
parameters? With 8 ro 10 parameters I would do something like this:
create table test_log(country varchar(25), city varchar(25))
go
create procedure myproc(@.country varchar(25), @.city varchar(25))
as
insert into test_log values(@.country, @.city)
go
declare @.country varchar(25), @.city varchar(25)
declare test_cases cursor
for
select * from
(
-- more than 50% customers
select 'USA' country_name
union all
-- less then 1% customers
select 'New Zealand'
union all
select NULL) country,
(select 'Boston' city
union all
-- city inconsistent with any country from the list above
select 'Kharkiv' city
union all
select null
) city
open test_cases
fetch next from test_cases into @.country, @.city
while @.@.fetch_status=0
begin
exec myproc @.country, @.city
fetch next from test_cases into @.country, @.city
end
go
select * from test_log
country city
-- --
USA Boston
USA Kharkiv
USA NULL
New Zealand Boston
New Zealand Kharkiv
New Zealand NULL
NULL Boston
NULL Kharkiv
NULL NULL
(9 row(s) affected)
go
drop table test_log
drop procedure myproc
So, for 2 parameters I needed 9 calls to do a unit test. Of course,
there is no need to open a cursor for mere 9 calls, I just wanted to
demostrate the technique used to make 1K calls.
Are you making 1 million calls for unit testing of your procedure with
20 parameters?
|||This is a classic example where dynamic execution should be considered.
You can find details here, assuming you have a subscription to SQLMag:
http://www.windowsitpro.com/Article/...502/47502.html
If you don't, let me know and I'll try to summarize.
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
> Hi,
> I have created the following SP and indexes, but the execution plan for SP
> shows that query optimizer always uses 'index scan'.
> --
> use northwind
> go
> create proc usp_search
> @.country varchar(100)=null,
> @.city varchar(100)=null
> as
> select customerid,companyname,country,city from customers where
> (@.country is null OR country=@.country)
> and
> (@.city is null OR city=@.city)
> go
> create index ix10 on customers(country,city,companyname)
> create index ix11 on customers(city,country,companyname)
> go
> exec usp_search 'uk','london' with recompile
> --
>
> If I remove any of the ORs, one of my indxes will be used. Are there any
> solution to keep both ORs and optimizer uses my indexes? Should I force
> optimizer to use any index?
> Thanks in advance,
> Leila
>
>
|||Thanks Itzik,
I'll be most grateful if you could do that.
BTW, what's your idea about this manner:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
It works very good and performs an 'Index Seek'. But one of the problems
that I noticed is on numeric columns(parameters). It has to implicitly
convert the number to varchar, so it doesn't perform Index seek, rather it
does Index Scan. I mean in the worst situation, its performance is like the
SP which I wrote in my first post (using NULLs)
Leila
"Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in message
news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
> This is a classic example where dynamic execution should be considered.
> You can find details here, assuming you have a subscription to SQLMag:
> http://www.windowsitpro.com/Article/...502/47502.html
> If you don't, let me know and I'll try to summarize.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>
> "Leila" <Leilas@.hotpop.com> wrote in message
> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>
|||If you had followed any ISO standards instead of what you made up on
the fly, would it look more like this? Without the super long
parameters that invite errors? With an ISO-11179 names?
CREATE PROCEDURE SearchCity
(@.my_country_code CHAR(3) = NULL, -- ISO standards!!
@.my_city_naem VARCHAR(25) = NULL -- postal union standards
AS
SELECT customer_id, company_name, country_code, city_name
FROM Customers
WHERE COALESCE (@.my_country_code, country_code = country_code)
AND COALESCE (@.my_ city_name, city_name) = city_anme ;
Since you never thought to post DDL, can we assume that (company_name,
city_name, country_code) is the key? The usual rule is to order an
index by the most selective to the least selective column.
SQL Server's optimizer is still a bit behind, so it the COALESCE()
trick does not work as well as it does in other products, such as DB2,
that can spot this form. I am not sure if SQL-2005 can do it.
|||Thanks Joe,
The COALESCE function (in SQL Server 2005) produces the same execution plan
as using IS NULL manner (an index scan is performed). But using '%' and
'like' performs index seek when you use it to seach strings:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegr oups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>
|||Again you show your complete lack of real world implementation experience of
SQL.
The query you present will give a table or index scan and will not scale, if
will cause SIGNIFICANT performance problems on a large table.
You should use IF ELSE at the very least to code for each optional parameter
combination, this can be done in the stored procedure, a single stored
procedure without having to bloat code and go for multiple stored procedures
which would lead to a more complicated design and increase your development
and maintanence costs.
Tony Rogerson
SQL Server MVP
http://sqlserverfaq.com - free video tutorials
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegr oups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>

Optimizing a query

Hi,
I have created the following SP and indexes, but the execution plan for SP
shows that query optimizer always uses 'index scan'.
--
use northwind
go
create proc usp_search
@.country varchar(100)=null,
@.city varchar(100)=null
as
select customerid,companyname,country,city from customers where
(@.country is null OR country=@.country)
and
(@.city is null OR city=@.city)
go
create index ix10 on customers(country,city,companyname)
create index ix11 on customers(city,country,companyname)
go
exec usp_search 'uk','london' with recompile
--
If I remove any of the ORs, one of my indxes will be used. Are there any
solution to keep both ORs and optimizer uses my indexes? Should I force
optimizer to use any index?
Thanks in advance,
LeilaLeila,
in this case it helps to be specific. Since you only have 4 cases, a
nested IF ... ELSE will do the trick
*untested*:
if (@.country is null)
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
end
end
else
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
where country=@.country
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
and country=@.country
end
end
Good luck!|||You might want to start from:
http://www.sommarskog.se/dyn-search.html
Anith|||Thanks Alexander,
But my real SP has 20 parameters. This SP was only a sample of what I want
to do.
"Alexander Kuznetsov" <AK_TIREDOFSPAM@.hotmail.COM> wrote in message
news:1138634724.575101.70310@.g44g2000cwa.googlegroups.com...
> Leila,
> in this case it helps to be specific. Since you only have 4 cases, a
> nested IF ... ELSE will do the trick
> *untested*:
> if (@.country is null)
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> end
> end
> else
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> where country=@.country
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> and country=@.country
> end
> end
> Good luck!
>|||Leila,
In that case I would concur with Anith. I would utilize dynamic SQL, as
it is described in Erland's article he mentioned.
Yet I have a question for you. How are you testing your SP with 20
parameters? With 8 ro 10 parameters I would do something like this:
create table test_log(country varchar(25), city varchar(25))
go
create procedure myproc(@.country varchar(25), @.city varchar(25))
as
insert into test_log values(@.country, @.city)
go
declare @.country varchar(25), @.city varchar(25)
declare test_cases cursor
for
select * from
(
-- more than 50% customers
select 'USA' country_name
union all
-- less then 1% customers
select 'New Zealand'
union all
select NULL) country,
(select 'Boston' city
union all
-- city inconsistent with any country from the list above
select 'Kharkiv' city
union all
select null
) city
open test_cases
fetch next from test_cases into @.country, @.city
while @.@.fetch_status=0
begin
exec myproc @.country, @.city
fetch next from test_cases into @.country, @.city
end
go
select * from test_log
country city
-- --
USA Boston
USA Kharkiv
USA NULL
New Zealand Boston
New Zealand Kharkiv
New Zealand NULL
NULL Boston
NULL Kharkiv
NULL NULL
(9 row(s) affected)
go
drop table test_log
drop procedure myproc
So, for 2 parameters I needed 9 calls to do a unit test. Of course,
there is no need to open a cursor for mere 9 calls, I just wanted to
demostrate the technique used to make 1K calls.
Are you making 1 million calls for unit testing of your procedure with
20 parameters?|||This is a classic example where dynamic execution should be considered.
You can find details here, assuming you have a subscription to SQLMag:
http://www.windowsitpro.com/Article...7502/47502.html
If you don't, let me know and I'll try to summarize.
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
> Hi,
> I have created the following SP and indexes, but the execution plan for SP
> shows that query optimizer always uses 'index scan'.
> --
> use northwind
> go
> create proc usp_search
> @.country varchar(100)=null,
> @.city varchar(100)=null
> as
> select customerid,companyname,country,city from customers where
> (@.country is null OR country=@.country)
> and
> (@.city is null OR city=@.city)
> go
> create index ix10 on customers(country,city,companyname)
> create index ix11 on customers(city,country,companyname)
> go
> exec usp_search 'uk','london' with recompile
> --
>
> If I remove any of the ORs, one of my indxes will be used. Are there any
> solution to keep both ORs and optimizer uses my indexes? Should I force
> optimizer to use any index?
> Thanks in advance,
> Leila
>
>|||Thanks Itzik,
I'll be most grateful if you could do that.
BTW, what's your idea about this manner:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
It works very good and performs an 'Index Seek'. But one of the problems
that I noticed is on numeric columns(parameters). It has to implicitly
convert the number to varchar, so it doesn't perform Index seek, rather it
does Index Scan. I mean in the worst situation, its performance is like the
SP which I wrote in my first post (using NULLs)
Leila
"Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in message
news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
> This is a classic example where dynamic execution should be considered.
> You can find details here, assuming you have a subscription to SQLMag:
> http://www.windowsitpro.com/Article...7502/47502.html
> If you don't, let me know and I'll try to summarize.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>
> "Leila" <Leilas@.hotpop.com> wrote in message
> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>|||If you had followed any ISO standards instead of what you made up on
the fly, would it look more like this? Without the super long
parameters that invite errors? With an ISO-11179 names?
CREATE PROCEDURE SearchCity
(@.my_country_code CHAR(3) = NULL, -- ISO standards!!
@.my_city_naem VARCHAR(25) = NULL -- postal union standards
AS
SELECT customer_id, company_name, country_code, city_name
FROM Customers
WHERE COALESCE (@.my_country_code, country_code = country_code)
AND COALESCE (@.my_ city_name, city_name) = city_anme ;
Since you never thought to post DDL, can we assume that (company_name,
city_name, country_code) is the key? The usual rule is to order an
index by the most selective to the least selective column.
SQL Server's optimizer is still a bit behind, so it the COALESCE()
trick does not work as well as it does in other products, such as DB2,
that can spot this form. I am not sure if SQL-2005 can do it.|||Thanks Joe,
The COALESCE function (in SQL Server 2005) produces the same execution plan
as using IS NULL manner (an index scan is performed). But using '%' and
'like' performs index seek when you use it to seach strings:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>|||Again you show your complete lack of real world implementation experience of
SQL.
The query you present will give a table or index scan and will not scale, if
will cause SIGNIFICANT performance problems on a large table.
You should use IF ELSE at the very least to code for each optional parameter
combination, this can be done in the stored procedure, a single stored
procedure without having to bloat code and go for multiple stored procedures
which would lead to a more complicated design and increase your development
and maintanence costs.
Tony Rogerson
SQL Server MVP
http://sqlserverfaq.com - free video tutorials
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>

Optimizing a query

Hi,
I have created the following SP and indexes, but the execution plan for SP
shows that query optimizer always uses 'index scan'.
--
use northwind
go
create proc usp_search
@.country varchar(100)=null,
@.city varchar(100)=null
as
select customerid,companyname,country,city from customers where
(@.country is null OR country=@.country)
and
(@.city is null OR city=@.city)
go
create index ix10 on customers(country,city,companyname)
create index ix11 on customers(city,country,companyname)
go
exec usp_search 'uk','london' with recompile
--
If I remove any of the ORs, one of my indxes will be used. Are there any
solution to keep both ORs and optimizer uses my indexes? Should I force
optimizer to use any index?
Thanks in advance,
LeilaLeila,
in this case it helps to be specific. Since you only have 4 cases, a
nested IF ... ELSE will do the trick
*untested*:
if (@.country is null)
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
end
end
else
begin
if @.city is null
begin
select customerid,companyname,country,city from customers
where country=@.country
end
else
begin
select customerid,companyname,country,city from customers
where city=@.city
and country=@.country
end
end
Good luck!|||You might want to start from:
http://www.sommarskog.se/dyn-search.html
--
Anith|||Thanks Alexander,
But my real SP has 20 parameters. This SP was only a sample of what I want
to do.
"Alexander Kuznetsov" <AK_TIREDOFSPAM@.hotmail.COM> wrote in message
news:1138634724.575101.70310@.g44g2000cwa.googlegroups.com...
> Leila,
> in this case it helps to be specific. Since you only have 4 cases, a
> nested IF ... ELSE will do the trick
> *untested*:
> if (@.country is null)
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> end
> end
> else
> begin
> if @.city is null
> begin
> select customerid,companyname,country,city from customers
> where country=@.country
> end
> else
> begin
> select customerid,companyname,country,city from customers
> where city=@.city
> and country=@.country
> end
> end
> Good luck!
>|||Leila,
In that case I would concur with Anith. I would utilize dynamic SQL, as
it is described in Erland's article he mentioned.
Yet I have a question for you. How are you testing your SP with 20
parameters? With 8 ro 10 parameters I would do something like this:
create table test_log(country varchar(25), city varchar(25))
go
create procedure myproc(@.country varchar(25), @.city varchar(25))
as
insert into test_log values(@.country, @.city)
go
declare @.country varchar(25), @.city varchar(25)
declare test_cases cursor
for
select * from
(
-- more than 50% customers
select 'USA' country_name
union all
-- less then 1% customers
select 'New Zealand'
union all
select NULL) country,
(select 'Boston' city
union all
-- city inconsistent with any country from the list above
select 'Kharkiv' city
union all
select null
) city
open test_cases
fetch next from test_cases into @.country, @.city
while @.@.fetch_status=0
begin
exec myproc @.country, @.city
fetch next from test_cases into @.country, @.city
end
go
select * from test_log
country city
-- --
USA Boston
USA Kharkiv
USA NULL
New Zealand Boston
New Zealand Kharkiv
New Zealand NULL
NULL Boston
NULL Kharkiv
NULL NULL
(9 row(s) affected)
go
drop table test_log
drop procedure myproc
So, for 2 parameters I needed 9 calls to do a unit test. Of course,
there is no need to open a cursor for mere 9 calls, I just wanted to
demostrate the technique used to make 1K calls.
Are you making 1 million calls for unit testing of your procedure with
20 parameters?|||This is a classic example where dynamic execution should be considered.
You can find details here, assuming you have a subscription to SQLMag:
http://www.windowsitpro.com/Article/ArticleID/47502/47502.html
If you don't, let me know and I'll try to summarize.
--
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
> Hi,
> I have created the following SP and indexes, but the execution plan for SP
> shows that query optimizer always uses 'index scan'.
> --
> use northwind
> go
> create proc usp_search
> @.country varchar(100)=null,
> @.city varchar(100)=null
> as
> select customerid,companyname,country,city from customers where
> (@.country is null OR country=@.country)
> and
> (@.city is null OR city=@.city)
> go
> create index ix10 on customers(country,city,companyname)
> create index ix11 on customers(city,country,companyname)
> go
> exec usp_search 'uk','london' with recompile
> --
>
> If I remove any of the ORs, one of my indxes will be used. Are there any
> solution to keep both ORs and optimizer uses my indexes? Should I force
> optimizer to use any index?
> Thanks in advance,
> Leila
>
>|||Thanks Itzik,
I'll be most grateful if you could do that.
BTW, what's your idea about this manner:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
It works very good and performs an 'Index Seek'. But one of the problems
that I noticed is on numeric columns(parameters). It has to implicitly
convert the number to varchar, so it doesn't perform Index seek, rather it
does Index Scan. I mean in the worst situation, its performance is like the
SP which I wrote in my first post (using NULLs)
Leila
"Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in message
news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
> This is a classic example where dynamic execution should be considered.
> You can find details here, assuming you have a subscription to SQLMag:
> http://www.windowsitpro.com/Article/ArticleID/47502/47502.html
> If you don't, let me know and I'll try to summarize.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>
> "Leila" <Leilas@.hotpop.com> wrote in message
> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>> Hi,
>> I have created the following SP and indexes, but the execution plan for
>> SP shows that query optimizer always uses 'index scan'.
>> --
>> use northwind
>> go
>> create proc usp_search
>> @.country varchar(100)=null,
>> @.city varchar(100)=null
>> as
>> select customerid,companyname,country,city from customers where
>> (@.country is null OR country=@.country)
>> and
>> (@.city is null OR city=@.city)
>> go
>> create index ix10 on customers(country,city,companyname)
>> create index ix11 on customers(city,country,companyname)
>> go
>> exec usp_search 'uk','london' with recompile
>> --
>>
>> If I remove any of the ORs, one of my indxes will be used. Are there any
>> solution to keep both ORs and optimizer uses my indexes? Should I force
>> optimizer to use any index?
>> Thanks in advance,
>> Leila
>>
>|||If you had followed any ISO standards instead of what you made up on
the fly, would it look more like this? Without the super long
parameters that invite errors? With an ISO-11179 names?
CREATE PROCEDURE SearchCity
(@.my_country_code CHAR(3) = NULL, -- ISO standards!!
@.my_city_naem VARCHAR(25) = NULL -- postal union standards
AS
SELECT customer_id, company_name, country_code, city_name
FROM Customers
WHERE COALESCE (@.my_country_code, country_code = country_code)
AND COALESCE (@.my_ city_name, city_name) = city_anme ;
Since you never thought to post DDL, can we assume that (company_name,
city_name, country_code) is the key? The usual rule is to order an
index by the most selective to the least selective column.
SQL Server's optimizer is still a bit behind, so it the COALESCE()
trick does not work as well as it does in other products, such as DB2,
that can spot this form. I am not sure if SQL-2005 can do it.|||Thanks Joe,
The COALESCE function (in SQL Server 2005) produces the same execution plan
as using IS NULL manner (an index scan is performed). But using '%' and
'like' performs index seek when you use it to seach strings:
use AdventureWorks
go
create index ix1 on person.contact(LastName,FirstName,MiddleName)
create index ix2 on person.contact(FirstName,LastName,MiddleName)
go
create proc usp_02
@.LastName varchar(100)='%',
@.FirstName varchar(100)='%'
AS
SELECT MiddleName,LastName,FirstName from person.contact
where (LastName like @.LastName)
and
(FirstName like @.FirstName)
go
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>|||Again you show your complete lack of real world implementation experience of
SQL.
The query you present will give a table or index scan and will not scale, if
will cause SIGNIFICANT performance problems on a large table.
You should use IF ELSE at the very least to code for each optional parameter
combination, this can be done in the stored procedure, a single stored
procedure without having to bloat code and go for multiple stored procedures
which would lead to a more complicated design and increase your development
and maintanence costs.
--
Tony Rogerson
SQL Server MVP
http://sqlserverfaq.com - free video tutorials
"--CELKO--" <jcelko212@.earthlink.net> wrote in message
news:1138675949.716153.137090@.o13g2000cwo.googlegroups.com...
> If you had followed any ISO standards instead of what you made up on
> the fly, would it look more like this? Without the super long
> parameters that invite errors? With an ISO-11179 names?
> CREATE PROCEDURE SearchCity
> (@.my_country_code CHAR(3) = NULL, -- ISO standards!!
> @.my_city_naem VARCHAR(25) = NULL -- postal union standards
> AS
> SELECT customer_id, company_name, country_code, city_name
> FROM Customers
> WHERE COALESCE (@.my_country_code, country_code = country_code)
> AND COALESCE (@.my_ city_name, city_name) = city_anme ;
> Since you never thought to post DDL, can we assume that (company_name,
> city_name, country_code) is the key? The usual rule is to order an
> index by the most selective to the least selective column.
> SQL Server's optimizer is still a bit behind, so it the COALESCE()
> trick does not work as well as it does in other products, such as DB2,
> that can spot this form. I am not sure if SQL-2005 can do it.
>|||Sure,
I can make it really short. All solutions have flaws. ;-)
Seriously now; the static solutions include (as you already gathered by
now):
1. col = @.param OR @.param IS NULL
2. col = COALESCE(@.param, col)
3. col LIKE @.param
4. A series of IF statements
And probably others...
1 and 2 simply often yield inadequate query plans. The reason is that the
optimizer currently doesn't have the logic to develop different branches of
execution based on whether the input was or wasn't NULL, and invoking the
relevant ones based on the input in practice.
3 is limited to character strings.
4 is hard to maintain, and becomes harder as the number of parameters grows
larger (simple combinatorial exercise). Though, interestingly, you could
develop code using dynamic execution that auto-creates multiple stored
procedures, each with a static query in charge of a different combination of
supplied values, and one navigating/redirecting stored procedure.
As for a pure dynamic solution; if you develop it wisely, it beats all the
rest in terms of performance. Though it has the known drawbacks involved
with dynamic execution (SQL Injection, ugly code, and so on).
Here's an example of how the solution utilizing dynamic execution might look
like (note that I didn't include input validation, treatment of SQL
Injection attempts, exception handling):
USE Northwind;
GO
CREATE PROC dbo.usp_GetOrders
@.OrderID AS INT = NULL,
@.CustomerID AS NCHAR(5) = NULL,
@.EmployeeID AS INT = NULL,
@.OrderDate AS DATETIME = NULL
AS
DECLARE @.sql AS NVARCHAR(4000);
SET @.sql = N'SELECT OrderID, CustomerID, EmployeeID, OrderDate'
+ N' FROM dbo.Orders'
+ N' WHERE 1 = 1'
+ CASE WHEN @.OrderID IS NOT NULL THEN
N' AND OrderID = @.oid' ELSE N'' END
+ CASE WHEN @.CustomerID IS NOT NULL THEN
N' AND CustomerID = @.cid' ELSE N'' END
+ CASE WHEN @.EmployeeID IS NOT NULL THEN
N' AND EmployeeID = @.eid' ELSE N'' END
+ CASE WHEN @.OrderDate IS NOT NULL THEN
N' AND OrderDate = @.dt' ELSE N'' END;
EXEC sp_executesql
@.sql,
N'@.oid AS INT, @.cid AS NCHAR(5), @.eid AS INT, @.dt AS DATETIME',
@.oid = @.OrderID,
@.cid = @.CustomerID,
@.eid = @.EmployeeID,
@.dt = @.OrderDate;
GO
-- Test proc
EXEC dbo.usp_GetOrders @.OrderID = 10248;
EXEC dbo.usp_GetOrders @.OrderDate = '19970101';
EXEC dbo.usp_GetOrders @.CustomerID = N'CENTC';
EXEC dbo.usp_GetOrders @.EmployeeID = 5;
The trick here is that the same code string will be generated for the same
input parameter lists. This means that the solution will be able to reuse
execution plans for invocations with the same input parameter lists. You can
easily observe this by querying master.dbo.syscacheobjects.
--
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:eQYMdzeJGHA.668@.TK2MSFTNGP11.phx.gbl...
> Thanks Itzik,
> I'll be most grateful if you could do that.
> BTW, what's your idea about this manner:
> use AdventureWorks
> go
> create index ix1 on person.contact(LastName,FirstName,MiddleName)
> create index ix2 on person.contact(FirstName,LastName,MiddleName)
> go
> create proc usp_02
> @.LastName varchar(100)='%',
> @.FirstName varchar(100)='%'
> AS
> SELECT MiddleName,LastName,FirstName from person.contact
> where (LastName like @.LastName)
> and
> (FirstName like @.FirstName)
> go
> It works very good and performs an 'Index Seek'. But one of the problems
> that I noticed is on numeric columns(parameters). It has to implicitly
> convert the number to varchar, so it doesn't perform Index seek, rather it
> does Index Scan. I mean in the worst situation, its performance is like
> the SP which I wrote in my first post (using NULLs)
> Leila
>
>
> "Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in
> message news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
>> This is a classic example where dynamic execution should be considered.
>> You can find details here, assuming you have a subscription to SQLMag:
>> http://www.windowsitpro.com/Article/ArticleID/47502/47502.html
>> If you don't, let me know and I'll try to summarize.
>> --
>> BG, SQL Server MVP
>> www.SolidQualityLearning.com
>> www.insidetsql.com
>> Anything written in this message represents my view, my own view, and
>> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>>
>> "Leila" <Leilas@.hotpop.com> wrote in message
>> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>> Hi,
>> I have created the following SP and indexes, but the execution plan for
>> SP shows that query optimizer always uses 'index scan'.
>> --
>> use northwind
>> go
>> create proc usp_search
>> @.country varchar(100)=null,
>> @.city varchar(100)=null
>> as
>> select customerid,companyname,country,city from customers where
>> (@.country is null OR country=@.country)
>> and
>> (@.city is null OR city=@.city)
>> go
>> create index ix10 on customers(country,city,companyname)
>> create index ix11 on customers(city,country,companyname)
>> go
>> exec usp_search 'uk','london' with recompile
>> --
>>
>> If I remove any of the ORs, one of my indxes will be used. Are there any
>> solution to keep both ORs and optimizer uses my indexes? Should I force
>> optimizer to use any index?
>> Thanks in advance,
>> Leila
>>
>>
>|||Thanks indeed,
Will I need to use EXEC ... WITH RECOMPILE each time or the SP will be
recompiled when the supplied input parameters change? Will the SQL Server
keep the plan for series of parameters or over writes the plan each time?
"Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in message
news:%23VsH2NoJGHA.2912@.tk2msftngp13.phx.gbl...
> Sure,
> I can make it really short. All solutions have flaws. ;-)
> Seriously now; the static solutions include (as you already gathered by
> now):
> 1. col = @.param OR @.param IS NULL
> 2. col = COALESCE(@.param, col)
> 3. col LIKE @.param
> 4. A series of IF statements
> And probably others...
> 1 and 2 simply often yield inadequate query plans. The reason is that the
> optimizer currently doesn't have the logic to develop different branches
> of execution based on whether the input was or wasn't NULL, and invoking
> the relevant ones based on the input in practice.
> 3 is limited to character strings.
> 4 is hard to maintain, and becomes harder as the number of parameters
> grows larger (simple combinatorial exercise). Though, interestingly, you
> could develop code using dynamic execution that auto-creates multiple
> stored procedures, each with a static query in charge of a different
> combination of supplied values, and one navigating/redirecting stored
> procedure.
> As for a pure dynamic solution; if you develop it wisely, it beats all the
> rest in terms of performance. Though it has the known drawbacks involved
> with dynamic execution (SQL Injection, ugly code, and so on).
> Here's an example of how the solution utilizing dynamic execution might
> look like (note that I didn't include input validation, treatment of SQL
> Injection attempts, exception handling):
> USE Northwind;
> GO
> CREATE PROC dbo.usp_GetOrders
> @.OrderID AS INT = NULL,
> @.CustomerID AS NCHAR(5) = NULL,
> @.EmployeeID AS INT = NULL,
> @.OrderDate AS DATETIME = NULL
> AS
> DECLARE @.sql AS NVARCHAR(4000);
> SET @.sql => N'SELECT OrderID, CustomerID, EmployeeID, OrderDate'
> + N' FROM dbo.Orders'
> + N' WHERE 1 = 1'
> + CASE WHEN @.OrderID IS NOT NULL THEN
> N' AND OrderID = @.oid' ELSE N'' END
> + CASE WHEN @.CustomerID IS NOT NULL THEN
> N' AND CustomerID = @.cid' ELSE N'' END
> + CASE WHEN @.EmployeeID IS NOT NULL THEN
> N' AND EmployeeID = @.eid' ELSE N'' END
> + CASE WHEN @.OrderDate IS NOT NULL THEN
> N' AND OrderDate = @.dt' ELSE N'' END;
> EXEC sp_executesql
> @.sql,
> N'@.oid AS INT, @.cid AS NCHAR(5), @.eid AS INT, @.dt AS DATETIME',
> @.oid = @.OrderID,
> @.cid = @.CustomerID,
> @.eid = @.EmployeeID,
> @.dt = @.OrderDate;
> GO
> -- Test proc
> EXEC dbo.usp_GetOrders @.OrderID = 10248;
> EXEC dbo.usp_GetOrders @.OrderDate = '19970101';
> EXEC dbo.usp_GetOrders @.CustomerID = N'CENTC';
> EXEC dbo.usp_GetOrders @.EmployeeID = 5;
> The trick here is that the same code string will be generated for the same
> input parameter lists. This means that the solution will be able to reuse
> execution plans for invocations with the same input parameter lists. You
> can easily observe this by querying master.dbo.syscacheobjects.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>
> "Leila" <Leilas@.hotpop.com> wrote in message
> news:eQYMdzeJGHA.668@.TK2MSFTNGP11.phx.gbl...
>> Thanks Itzik,
>> I'll be most grateful if you could do that.
>> BTW, what's your idea about this manner:
>> use AdventureWorks
>> go
>> create index ix1 on person.contact(LastName,FirstName,MiddleName)
>> create index ix2 on person.contact(FirstName,LastName,MiddleName)
>> go
>> create proc usp_02
>> @.LastName varchar(100)='%',
>> @.FirstName varchar(100)='%'
>> AS
>> SELECT MiddleName,LastName,FirstName from person.contact
>> where (LastName like @.LastName)
>> and
>> (FirstName like @.FirstName)
>> go
>> It works very good and performs an 'Index Seek'. But one of the problems
>> that I noticed is on numeric columns(parameters). It has to implicitly
>> convert the number to varchar, so it doesn't perform Index seek, rather
>> it does Index Scan. I mean in the worst situation, its performance is
>> like the SP which I wrote in my first post (using NULLs)
>> Leila
>>
>>
>> "Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in
>> message news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
>> This is a classic example where dynamic execution should be considered.
>> You can find details here, assuming you have a subscription to SQLMag:
>> http://www.windowsitpro.com/Article/ArticleID/47502/47502.html
>> If you don't, let me know and I'll try to summarize.
>> --
>> BG, SQL Server MVP
>> www.SolidQualityLearning.com
>> www.insidetsql.com
>> Anything written in this message represents my view, my own view, and
>> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>>
>> "Leila" <Leilas@.hotpop.com> wrote in message
>> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>> Hi,
>> I have created the following SP and indexes, but the execution plan for
>> SP shows that query optimizer always uses 'index scan'.
>> --
>> use northwind
>> go
>> create proc usp_search
>> @.country varchar(100)=null,
>> @.city varchar(100)=null
>> as
>> select customerid,companyname,country,city from customers where
>> (@.country is null OR country=@.country)
>> and
>> (@.city is null OR city=@.city)
>> go
>> create index ix10 on customers(country,city,companyname)
>> create index ix11 on customers(city,country,companyname)
>> go
>> exec usp_search 'uk','london' with recompile
>> --
>>
>> If I remove any of the ORs, one of my indxes will be used. Are there
>> any solution to keep both ORs and optimizer uses my indexes? Should I
>> force optimizer to use any index?
>> Thanks in advance,
>> Leila
>>
>>
>>
>|||Just a short with respect to
> 3. col LIKE @.param
> 3 is limited to character strings.
Yes, LIKE will only work for (n)(var)char. But for other data types, the
combination of COALESCE and BETWEEN can be used. For an int, this could
be
col BETWEEN COALESCE(@.param, -2147483648) AND COALESCE(@.param,
2147483647)
For a smalldatetime, this could be
col BETWEEN COALESCE(@.param, '19000101') AND COALESCE(@.param,
'20790606 23:59')
Etcetera.
Gert-Jan
Itzik Ben-Gan wrote:
> Sure,
> I can make it really short. All solutions have flaws. ;-)
> Seriously now; the static solutions include (as you already gathered by
> now):
> 1. col = @.param OR @.param IS NULL
> 2. col = COALESCE(@.param, col)
> 3. col LIKE @.param
> 4. A series of IF statements
> And probably others...
> 1 and 2 simply often yield inadequate query plans. The reason is that the
> optimizer currently doesn't have the logic to develop different branches of
> execution based on whether the input was or wasn't NULL, and invoking the
> relevant ones based on the input in practice.
> 3 is limited to character strings.
> 4 is hard to maintain, and becomes harder as the number of parameters grows
> larger (simple combinatorial exercise). Though, interestingly, you could
> develop code using dynamic execution that auto-creates multiple stored
> procedures, each with a static query in charge of a different combination of
> supplied values, and one navigating/redirecting stored procedure.
> As for a pure dynamic solution; if you develop it wisely, it beats all the
> rest in terms of performance. Though it has the known drawbacks involved
> with dynamic execution (SQL Injection, ugly code, and so on).
> Here's an example of how the solution utilizing dynamic execution might look
> like (note that I didn't include input validation, treatment of SQL
> Injection attempts, exception handling):
> USE Northwind;
> GO
> CREATE PROC dbo.usp_GetOrders
> @.OrderID AS INT = NULL,
> @.CustomerID AS NCHAR(5) = NULL,
> @.EmployeeID AS INT = NULL,
> @.OrderDate AS DATETIME = NULL
> AS
> DECLARE @.sql AS NVARCHAR(4000);
> SET @.sql => N'SELECT OrderID, CustomerID, EmployeeID, OrderDate'
> + N' FROM dbo.Orders'
> + N' WHERE 1 = 1'
> + CASE WHEN @.OrderID IS NOT NULL THEN
> N' AND OrderID = @.oid' ELSE N'' END
> + CASE WHEN @.CustomerID IS NOT NULL THEN
> N' AND CustomerID = @.cid' ELSE N'' END
> + CASE WHEN @.EmployeeID IS NOT NULL THEN
> N' AND EmployeeID = @.eid' ELSE N'' END
> + CASE WHEN @.OrderDate IS NOT NULL THEN
> N' AND OrderDate = @.dt' ELSE N'' END;
> EXEC sp_executesql
> @.sql,
> N'@.oid AS INT, @.cid AS NCHAR(5), @.eid AS INT, @.dt AS DATETIME',
> @.oid = @.OrderID,
> @.cid = @.CustomerID,
> @.eid = @.EmployeeID,
> @.dt = @.OrderDate;
> GO
> -- Test proc
> EXEC dbo.usp_GetOrders @.OrderID = 10248;
> EXEC dbo.usp_GetOrders @.OrderDate = '19970101';
> EXEC dbo.usp_GetOrders @.CustomerID = N'CENTC';
> EXEC dbo.usp_GetOrders @.EmployeeID = 5;
> The trick here is that the same code string will be generated for the same
> input parameter lists. This means that the solution will be able to reuse
> execution plans for invocations with the same input parameter lists. You can
> easily observe this by querying master.dbo.syscacheobjects.
> --
> BG, SQL Server MVP
> www.SolidQualityLearning.com
> www.insidetsql.com
> Anything written in this message represents my view, my own view, and
> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
[snip]|||Here's the beauty--no need to create or execute the proc WITH RECOMPILE.
Dynamic execution operates in a separate batch than the outer level's batch
(the proc's batch in this case), meaning that the dynamic batch is parsed
and optimized separately.
This fact may sometimes be a disadvantage, but in our case it is an
advantage; each unique code string (one per unique parameters list) will
yield a separate execution plan, which will be reused only by the same code
string invoked again. You will end up with as many plans as the unique
parameter lists used in practice.
I suggested querying master.dbo.syscacheobjects to witness this behavior.
--
BG, SQL Server MVP
www.SolidQualityLearning.com
www.insidetsql.com
Anything written in this message represents my view, my own view, and
nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
"Leila" <Leilas@.hotpop.com> wrote in message
news:OZUgtJpJGHA.3896@.TK2MSFTNGP15.phx.gbl...
> Thanks indeed,
> Will I need to use EXEC ... WITH RECOMPILE each time or the SP will be
> recompiled when the supplied input parameters change? Will the SQL Server
> keep the plan for series of parameters or over writes the plan each time?
>
> "Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in
> message news:%23VsH2NoJGHA.2912@.tk2msftngp13.phx.gbl...
>> Sure,
>> I can make it really short. All solutions have flaws. ;-)
>> Seriously now; the static solutions include (as you already gathered by
>> now):
>> 1. col = @.param OR @.param IS NULL
>> 2. col = COALESCE(@.param, col)
>> 3. col LIKE @.param
>> 4. A series of IF statements
>> And probably others...
>> 1 and 2 simply often yield inadequate query plans. The reason is that the
>> optimizer currently doesn't have the logic to develop different branches
>> of execution based on whether the input was or wasn't NULL, and invoking
>> the relevant ones based on the input in practice.
>> 3 is limited to character strings.
>> 4 is hard to maintain, and becomes harder as the number of parameters
>> grows larger (simple combinatorial exercise). Though, interestingly, you
>> could develop code using dynamic execution that auto-creates multiple
>> stored procedures, each with a static query in charge of a different
>> combination of supplied values, and one navigating/redirecting stored
>> procedure.
>> As for a pure dynamic solution; if you develop it wisely, it beats all
>> the rest in terms of performance. Though it has the known drawbacks
>> involved with dynamic execution (SQL Injection, ugly code, and so on).
>> Here's an example of how the solution utilizing dynamic execution might
>> look like (note that I didn't include input validation, treatment of SQL
>> Injection attempts, exception handling):
>> USE Northwind;
>> GO
>> CREATE PROC dbo.usp_GetOrders
>> @.OrderID AS INT = NULL,
>> @.CustomerID AS NCHAR(5) = NULL,
>> @.EmployeeID AS INT = NULL,
>> @.OrderDate AS DATETIME = NULL
>> AS
>> DECLARE @.sql AS NVARCHAR(4000);
>> SET @.sql =>> N'SELECT OrderID, CustomerID, EmployeeID, OrderDate'
>> + N' FROM dbo.Orders'
>> + N' WHERE 1 = 1'
>> + CASE WHEN @.OrderID IS NOT NULL THEN
>> N' AND OrderID = @.oid' ELSE N'' END
>> + CASE WHEN @.CustomerID IS NOT NULL THEN
>> N' AND CustomerID = @.cid' ELSE N'' END
>> + CASE WHEN @.EmployeeID IS NOT NULL THEN
>> N' AND EmployeeID = @.eid' ELSE N'' END
>> + CASE WHEN @.OrderDate IS NOT NULL THEN
>> N' AND OrderDate = @.dt' ELSE N'' END;
>> EXEC sp_executesql
>> @.sql,
>> N'@.oid AS INT, @.cid AS NCHAR(5), @.eid AS INT, @.dt AS DATETIME',
>> @.oid = @.OrderID,
>> @.cid = @.CustomerID,
>> @.eid = @.EmployeeID,
>> @.dt = @.OrderDate;
>> GO
>> -- Test proc
>> EXEC dbo.usp_GetOrders @.OrderID = 10248;
>> EXEC dbo.usp_GetOrders @.OrderDate = '19970101';
>> EXEC dbo.usp_GetOrders @.CustomerID = N'CENTC';
>> EXEC dbo.usp_GetOrders @.EmployeeID = 5;
>> The trick here is that the same code string will be generated for the
>> same input parameter lists. This means that the solution will be able to
>> reuse execution plans for invocations with the same input parameter
>> lists. You can easily observe this by querying
>> master.dbo.syscacheobjects.
>> --
>> BG, SQL Server MVP
>> www.SolidQualityLearning.com
>> www.insidetsql.com
>> Anything written in this message represents my view, my own view, and
>> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>>
>> "Leila" <Leilas@.hotpop.com> wrote in message
>> news:eQYMdzeJGHA.668@.TK2MSFTNGP11.phx.gbl...
>> Thanks Itzik,
>> I'll be most grateful if you could do that.
>> BTW, what's your idea about this manner:
>> use AdventureWorks
>> go
>> create index ix1 on person.contact(LastName,FirstName,MiddleName)
>> create index ix2 on person.contact(FirstName,LastName,MiddleName)
>> go
>> create proc usp_02
>> @.LastName varchar(100)='%',
>> @.FirstName varchar(100)='%'
>> AS
>> SELECT MiddleName,LastName,FirstName from person.contact
>> where (LastName like @.LastName)
>> and
>> (FirstName like @.FirstName)
>> go
>> It works very good and performs an 'Index Seek'. But one of the problems
>> that I noticed is on numeric columns(parameters). It has to implicitly
>> convert the number to varchar, so it doesn't perform Index seek, rather
>> it does Index Scan. I mean in the worst situation, its performance is
>> like the SP which I wrote in my first post (using NULLs)
>> Leila
>>
>>
>> "Itzik Ben-Gan" <itzik@.REMOVETHIS.SolidQualityLearning.com> wrote in
>> message news:eSfAJRdJGHA.1028@.TK2MSFTNGP11.phx.gbl...
>> This is a classic example where dynamic execution should be considered.
>> You can find details here, assuming you have a subscription to SQLMag:
>> http://www.windowsitpro.com/Article/ArticleID/47502/47502.html
>> If you don't, let me know and I'll try to summarize.
>> --
>> BG, SQL Server MVP
>> www.SolidQualityLearning.com
>> www.insidetsql.com
>> Anything written in this message represents my view, my own view, and
>> nothing but my view (WITH SCHEMABINDING), so help me my T-SQL code.
>>
>> "Leila" <Leilas@.hotpop.com> wrote in message
>> news:O1PW%23$aJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>> Hi,
>> I have created the following SP and indexes, but the execution plan
>> for SP shows that query optimizer always uses 'index scan'.
>> --
>> use northwind
>> go
>> create proc usp_search
>> @.country varchar(100)=null,
>> @.city varchar(100)=null
>> as
>> select customerid,companyname,country,city from customers where
>> (@.country is null OR country=@.country)
>> and
>> (@.city is null OR city=@.city)
>> go
>> create index ix10 on customers(country,city,companyname)
>> create index ix11 on customers(city,country,companyname)
>> go
>> exec usp_search 'uk','london' with recompile
>> --
>>
>> If I remove any of the ORs, one of my indxes will be used. Are there
>> any solution to keep both ORs and optimizer uses my indexes? Should I
>> force optimizer to use any index?
>> Thanks in advance,
>> Leila
>>
>>
>>
>>
>