I am working with a report generator that is based on SQL Server 2000 and
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-Gary
Gary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen
|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/pro.../c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
Showing posts with label basically. Show all posts
Showing posts with label basically. Show all posts
Monday, March 12, 2012
Optimal configuration for report generator
I am working with a report generator that is based on SQL Server 2000 and
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-Gary
Gary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen
|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/pro.../c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-Gary
Gary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen
|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/pro.../c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
Optimal configuration for report generator
I am working with a report generator that is based on SQL Server 2000 and
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-GaryGary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/prodtechnol/sql/2000/books/c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-GaryGary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/prodtechnol/sql/2000/books/c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
Optimal configuration for report generator
I am working with a report generator that is based on SQL Server 2000 and
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
uses ASP as the UI. Basically we have a set of reports that end users can
execute through a web browser. In general the model works fine, but we are
running into some scaling issues.
What I'm trying to determine is, what is the optimal configuration for this
system. It is currently a 2.4G Pentium with a large RAID and 1G of RAM. We
have been using the "fixed" memory configuration, allocating 864M to SQL.
This is on a Windows 2003 server box.
This works fine when a "small" query or two is executed, but the performance
suffers terribly when several users try to run reports in parallel. A single
query might take 10 minutes to run if nothing else is happening on the box,
but if additional users log on an run reports, it's almost impossible to
predict when the queries will finish.
I am also looking at the effect of database size on performance, running
tests against a database with 1 month, 3 months, and say 12 months of data,
running the same query against 2 databases in parallel. With the original
configuration, the results were all over the place, with the 12 month
database outperforming the smaller dbs, while other times there was little
difference. It seems that once the system starts paging, and paging heavily,
it's over; the system never "recovers" and queries that previously ran in a
few minutes now take hours.
I added 3 G more memory to the system, and modified boot.ini to include the
/3GB switch. Now when I run the same tests, the results are much more
consistent, as the system rarely ever has to swap. Then again I've never
seen it go past 1.7G in Task manager, making me think that any more than say
2.5G of memory is a waste?
Things we are trying to determine are:
- in the SQL Server memory configuration, is Fixed better than Dynamic? We
have read that Dynamic is not good at returning memory to the OS once it's
been allocated
- What else can we do to optimize the performance for this application? It
seems to me if the indexes are properly designed, the database size
shouldn't have that much impact on performance, but this appears to be true
only to a point. In comparing the execution plans between say a 12 month and
a 3 month database, the plans are sometimes dramatically different. I assume
this is due to the optimizer deciding that going directly to the base tables
and not using an index will result in better performance, when in reality,
this doesn't always appear to be true.
- Are there other SQL Server switches I should be tweaking? Is there some
number of simultaneous queries that this configuration should be limited to?
- What about other versions of SQL Server (e.g. Enterprise, Data Center,
etc) would these buy us anything?
Thanks for any advice,
-GaryGary,
I suppose I would have to have more information. Performance and Tuning
is a step by step process (one is always robbing Peter to pay Paul, so
to speak).
how big is the database in question?
what is the RAID configuration (0,1,5,10)?
what does the queries that are running look like? have you looked at
the execution plan to make sure the indexes are being used? (a nested
iteration join with a table scan can cause massive slow downs).
how big are the tables?
it might be the case that you are suffering from spinlock contention
(context switching on the processor)...in which case you might need to
have more processors. you can run a profile on the processor
performance to get the proper information. however, i would strongly
suggest looking at the logical design of the database, the query
itself, the statistics on the objects involved and the indexes. the
optimizer could be ignoring your indexes because the statistics are out
of wack or the weight of the search is to high.
there are just too many factors that could be responsible for the
slowness for me to answer without knowing them.
there is a decent performance and tuning section in the manual, if you
want to give it a look.
hth.
hans nelsen|||Here are my two cents. Mileage may vary, past results do not guarantee
future performance etc...
-- in the SQL Server memory configuration, is Fixed better than
Dynamic? We have read that Dynamic is not good at returning memory to
the OS once it's been allocated
CHOOSE DYNAMIC. MSSQL is meant to function on its own box. It will
take over all the resources. If you're running MSFT small business
server - then you might want fixed.
-- What else can we do to optimize the performance for this
application? It seems to me if the indexes are properly designed, the
database size shouldn't have that much impact on performance, but this
appears to be true only to a point. In comparing the execution plans
between say a 12 month and a 3 month database, the plans are sometimes
dramatically different. I assume this is due to the optimizer deciding
that going directly to the base tables
and not using an index will result in better performance, when in
reality,this doesn't always appear to be true.
COMMON PROBLEMS. The most common problems are user locks. Unlike
Oracle, MSSQL is lock crazy. It places locks on everything. Ensure
that read-locks are not competing against write-locks. Also ensure
that your indexes are being regularly defragged. DBCC DBREINDEX or
DBCC INDEXDEFRAG. Looking at the execution plan in query analyzer --
should tell you. Auto create statistics and auto update statistics
should also be turned on.
-- Are there other SQL Server switches I should be tweaking? Is there
some number of simultaneous queries that this configuration should be
limited to?
Look at this free article from microsoft/technet.
http://www.microsoft.com/technet/pr...s/c02ppcsq.mspx
Here is a quote from it:
Caution: Setting fixed memory incorrectly can cause serious performance
problems on SQL Server. Use fixed memory only in circumstances when you
need to ensure that an exact amount of memory is available for SQL
Server.
-- What about other versions of SQL Server (e.g. Enterprise, Data
Center, etc) would these buy us anything?
No. Standard works just fine for 99% of users. Enterprise does get
you Reporting Services and indexed views and log shipping.
Wednesday, March 7, 2012
operator precedence challenge
I'm having problems with the query below. It works fine until the "NOT IN" part and I'm not sure why. Basically, I'm getting records where their educ_audio field is set to "no". I've messed with the parens to try to force SQL to process the NOT IN part before the other clauses without avail. Can someone shed some light?
TIA
SELECT distinct contacts.fname, contacts.lname, contacts.company, contacts.contact_id, contacts.business_phone, contacts.emailAddress, contacts.dateLastContact FROM journal INNER JOIN contacts ON journal.contact_id = contacts.contact_id INNER JOIN products ON journal.product_code = products.product_code WHERE ( journal.product_code IN ('ABLE') ) OR (( journal.product_code IN ('JOBS') ) AND ( journal.product_status IN ('12','14','15') )) OR (( products.prod_design IN ('audio') ) AND ( products.library_code IN ('hrss') )) AND (journal.journal_id NOT IN (SELECT journal.journal_id FROM journal INNER JOIN contacts ON journal.contact_id = contacts.contact_id INNER JOIN products ON products.product_code = journal.product_code where ( contacts.educ_audio IN ('no') ) )) ORDER BY contacts.lname asc, contacts.fnameI think you can get the results you want by enclosing all your OR clauses in one set of parenthesis. If I understand correctly, your records must satisfy at least one of the OR clauses, and the AND clause:
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
WHERE ((journal.product_code IN ('ABLE'))
OR ((journal.product_code IN ('JOBS')) AND (journal.product_status IN ('12','14','15')))
OR ((products.prod_design IN ('audio')) AND (products.library_code IN ('hrss'))))
AND (journal.journal_id NOT IN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE (contacts.educ_audio IN ('no'))))
ORDER BY contacts.lname asc,
contacts.fname
But you can clean this up a lot more
First, IN ('ABLE') is equivalent to ='ABLE', so don't muddy the waters with more parentheses than you need.
Second, many of your Parenthesis pairs are superfluous, in that they enclose only one clause. Eliminate the clutter.
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
WHERE (journal.product_code = 'ABLE'
OR (journal.product_code = 'JOBS' AND journal.product_status IN ('12','14','15'))
OR (products.prod_design = 'audio' AND products.library_code IN ('hrss')))
AND journal.journal_id NOT IN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE contacts.educ_audio = 'no')
ORDER BY contacts.lname asc,
contacts.fname
--Lastly, consider converting you NOT IN clause to a LEFT OUTER JOIN subquery:
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
LEFT OUTER JOIN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE contacts.educ_audio = 'no') ExcludeRecords
on journal.journal_id = ExludeRecords.journal_id
WHERE (journal.product_code = 'ABLE'
OR (journal.product_code = 'JOBS' AND journal.product_status IN ('12','14','15'))
OR (products.prod_design = 'audio' AND products.library_code IN ('hrss')))
AND ExcludeRecords.journal_id is null
ORDER BY contacts.lname asc,
contacts.fname
If you take the time to develop a coding style that includes neat and consistent indenting and formatting, you will be rewarded with much clearer and more bug-free code.|||Thanks for your help.
Reason the SQL looks like it does (not indented, etc) is because that was a paste from my application. This is for an ad-hoc query tool I'm building.
As to your suggestions, they all seem to work except the last one.
Error: The column prefix 'ExludeRecords' does not match with a table name or alias name used in the query.|||Just a typo.
on journal.journal_id = ExludeRecords.journal_id
...should have been:
on journal.journal_id = ExcludeRecords.journal_id
TIA
SELECT distinct contacts.fname, contacts.lname, contacts.company, contacts.contact_id, contacts.business_phone, contacts.emailAddress, contacts.dateLastContact FROM journal INNER JOIN contacts ON journal.contact_id = contacts.contact_id INNER JOIN products ON journal.product_code = products.product_code WHERE ( journal.product_code IN ('ABLE') ) OR (( journal.product_code IN ('JOBS') ) AND ( journal.product_status IN ('12','14','15') )) OR (( products.prod_design IN ('audio') ) AND ( products.library_code IN ('hrss') )) AND (journal.journal_id NOT IN (SELECT journal.journal_id FROM journal INNER JOIN contacts ON journal.contact_id = contacts.contact_id INNER JOIN products ON products.product_code = journal.product_code where ( contacts.educ_audio IN ('no') ) )) ORDER BY contacts.lname asc, contacts.fnameI think you can get the results you want by enclosing all your OR clauses in one set of parenthesis. If I understand correctly, your records must satisfy at least one of the OR clauses, and the AND clause:
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
WHERE ((journal.product_code IN ('ABLE'))
OR ((journal.product_code IN ('JOBS')) AND (journal.product_status IN ('12','14','15')))
OR ((products.prod_design IN ('audio')) AND (products.library_code IN ('hrss'))))
AND (journal.journal_id NOT IN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE (contacts.educ_audio IN ('no'))))
ORDER BY contacts.lname asc,
contacts.fname
But you can clean this up a lot more
First, IN ('ABLE') is equivalent to ='ABLE', so don't muddy the waters with more parentheses than you need.
Second, many of your Parenthesis pairs are superfluous, in that they enclose only one clause. Eliminate the clutter.
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
WHERE (journal.product_code = 'ABLE'
OR (journal.product_code = 'JOBS' AND journal.product_status IN ('12','14','15'))
OR (products.prod_design = 'audio' AND products.library_code IN ('hrss')))
AND journal.journal_id NOT IN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE contacts.educ_audio = 'no')
ORDER BY contacts.lname asc,
contacts.fname
--Lastly, consider converting you NOT IN clause to a LEFT OUTER JOIN subquery:
SELECT distinct
contacts.fname,
contacts.lname,
contacts.company,
contacts.contact_id,
contacts.business_phone,
contacts.emailAddress,
contacts.dateLastContact
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON journal.product_code = products.product_code
LEFT OUTER JOIN
(SELECT journal.journal_id
FROM journal
INNER JOIN contacts ON journal.contact_id = contacts.contact_id
INNER JOIN products ON products.product_code = journal.product_code
WHERE contacts.educ_audio = 'no') ExcludeRecords
on journal.journal_id = ExludeRecords.journal_id
WHERE (journal.product_code = 'ABLE'
OR (journal.product_code = 'JOBS' AND journal.product_status IN ('12','14','15'))
OR (products.prod_design = 'audio' AND products.library_code IN ('hrss')))
AND ExcludeRecords.journal_id is null
ORDER BY contacts.lname asc,
contacts.fname
If you take the time to develop a coding style that includes neat and consistent indenting and formatting, you will be rewarded with much clearer and more bug-free code.|||Thanks for your help.
Reason the SQL looks like it does (not indented, etc) is because that was a paste from my application. This is for an ad-hoc query tool I'm building.
As to your suggestions, they all seem to work except the last one.
Error: The column prefix 'ExludeRecords' does not match with a table name or alias name used in the query.|||Just a typo.
on journal.journal_id = ExludeRecords.journal_id
...should have been:
on journal.journal_id = ExcludeRecords.journal_id
Operational and Analytical CRM??
Hi,
While going through an article I came across the subject terms and
wanted to ask the experts if these are "basically" the same thing as
"BI" or are these totally different?
I'll appreciate any pointers/help from anyone.
Thanks.
Dont' think they are the same. Here's my "guess"...
* CRM = Customer Relationsship Management - something about customers...but
in BI you can have lots of areas such as finance, sales, inventory and so
forth...
* however - i would guess that data mining could be relevant a lot in CRM on
websites and for that i would think an AS engine could be used
* In CRM there a many more processes related to customers - that could be
the operatonal CRM above and perhaps the analytical could be solved by
BI...?
"Learner" <wantnospam@.email.com> skrev i en meddelelse
news:MPG.1b1714e36d0ad7aa989703@.msnews.microsoft.c om...
> Hi,
> While going through an article I came across the subject terms and
> wanted to ask the experts if these are "basically" the same thing as
> "BI" or are these totally different?
> I'll appreciate any pointers/help from anyone.
> --
> Thanks.
While going through an article I came across the subject terms and
wanted to ask the experts if these are "basically" the same thing as
"BI" or are these totally different?
I'll appreciate any pointers/help from anyone.
Thanks.
Dont' think they are the same. Here's my "guess"...
* CRM = Customer Relationsship Management - something about customers...but
in BI you can have lots of areas such as finance, sales, inventory and so
forth...
* however - i would guess that data mining could be relevant a lot in CRM on
websites and for that i would think an AS engine could be used
* In CRM there a many more processes related to customers - that could be
the operatonal CRM above and perhaps the analytical could be solved by
BI...?
"Learner" <wantnospam@.email.com> skrev i en meddelelse
news:MPG.1b1714e36d0ad7aa989703@.msnews.microsoft.c om...
> Hi,
> While going through an article I came across the subject terms and
> wanted to ask the experts if these are "basically" the same thing as
> "BI" or are these totally different?
> I'll appreciate any pointers/help from anyone.
> --
> Thanks.
Operational and Analytical CRM??
Hi,
While going through an article I came across the subject terms and
wanted to ask the experts if these are "basically" the same thing as
"BI" or are these totally different?
I'll appreciate any pointers/help from anyone.
Thanks.Dont' think they are the same. Here's my "guess"...
* CRM = Customer Relationsship Management - something about customers...but
in BI you can have lots of areas such as finance, sales, inventory and so
forth...
* however - i would guess that data mining could be relevant a lot in CRM on
websites and for that i would think an AS engine could be used
* In CRM there a many more processes related to customers - that could be
the operatonal CRM above and perhaps the analytical could be solved by
BI...?
"Learner" <wantnospam@.email.com> skrev i en meddelelse
news:MPG.1b1714e36d0ad7aa989703@.msnews.microsoft.com...
> Hi,
> While going through an article I came across the subject terms and
> wanted to ask the experts if these are "basically" the same thing as
> "BI" or are these totally different?
> I'll appreciate any pointers/help from anyone.
> --
> Thanks.
While going through an article I came across the subject terms and
wanted to ask the experts if these are "basically" the same thing as
"BI" or are these totally different?
I'll appreciate any pointers/help from anyone.
Thanks.Dont' think they are the same. Here's my "guess"...
* CRM = Customer Relationsship Management - something about customers...but
in BI you can have lots of areas such as finance, sales, inventory and so
forth...
* however - i would guess that data mining could be relevant a lot in CRM on
websites and for that i would think an AS engine could be used
* In CRM there a many more processes related to customers - that could be
the operatonal CRM above and perhaps the analytical could be solved by
BI...?
"Learner" <wantnospam@.email.com> skrev i en meddelelse
news:MPG.1b1714e36d0ad7aa989703@.msnews.microsoft.com...
> Hi,
> While going through an article I came across the subject terms and
> wanted to ask the experts if these are "basically" the same thing as
> "BI" or are these totally different?
> I'll appreciate any pointers/help from anyone.
> --
> Thanks.
Subscribe to:
Posts (Atom)