OpenXML Identity Insert: application

Showing posts with label application. Show all posts

Friday, March 30, 2012

Optimizing table in mssql

Hi there,
Is there an optimize table command in mssql which will work the same way as "OPTIMIZE TABLE tablename" of mysql?

I have a php application that should work on both mysql and mssql. To do defragmentation, I am using the above command. Is there an equivalent in mssql?

Cheers,
CeliaI presume optimise table defragments the table and indexes? If not - please post what it means.

To defrag an index check out the following in BoL:
2000
- DBCC DBREINDEX
- DBCC INDEXDEFRAG
2005
- ALTER INDEX ... REORGANISE\ REBUILD

You will need to see which options are best for your environment.

Optimizing Queries generated at runtime

We are using SQL SERVER 2005, enterprise/standard edition for an application that generates queries at runtime and normally having many joins in it. But these queries are taking lot of time when they are executed. After setting the database parameter PARAMETERIZATION to FORCED, performance of the queries have improved a lot but still we want to improve them further. Is there any other parameter that we can set for improving the performance of the queries. We have created the indexes on the basic columns that will be used while querying but it may happen that some other columns may also be used for searching in which case queries become quiet slow. Is there anything like skip-scan indexes(of oracle) in sql server 2005.

Regards
Salil

Check if there are columns that participate in WHERE clause and have not indexes.

You say "an application that generates queries at runtime", try to use parametrized stored procedures that build dynamically queries using sp_executesql;

in queries don't use "select * from ..." use "select col1,col2,... from ..."

try to use WHERE clause in queries to force using indexes.

you say your queries have many joins on it, try to use indexed views, see article|||1. We have created the indexes on the main columns that we know will be included in the where clause but there can be some columns, we dont know, that can be included in the where clause.
2. Going for parameterized stored procedures is a nice option but we cant go for them at this stage of our product.
3. We are using column names in the select clause rather than "select * "
4. We have many joins but we dont know join will be made to which table.

Thats why we are looking for something more generic like the PARAMETERIZATION parameter. Is there anything else that we can set at the database level for improving the performance of our queries.

|||

Salil wrote:
1. We have created the indexes on the main columns that we know will be included in the where clause but there can be some columns, we dont know, that can be included in the where clause.
2. Going for parameterized stored procedures is a nice option but we cant go for them at this stage of our product.
3. We are using column names in the select clause rather than "select * "
4. We have many joins but we dont know join will be made to which table.

I think you work with a black box and you can optimize outside it; i think you haven't many choices

Check if database options related to STATISTICS are on "ON" option.

So, I think you have to open that black box....

|||we have already set AUTO_UPDATE_STATISTICS_ASYNC ON at the database level.

Wednesday, March 28, 2012

Optimizing Database

I have an application that's allows user input, and is translating it by
stripping out the html tags and also doing some code translations. The user
is able to later edit their input. However it's unfeasible to reverse
translate it back as the logic would be too complicated, and there are
instances where it won't be possible.

So, what I'm thinking to do to speed up performance is to duplicate the user
data, one for native data, and the other for the translated data. When user
edits their input, the native data is shown. When the application is
showing the data in a page, the translated data is shown.

My question is, would it make a performance difference if I store the native
data and the translated data in the same table, or would it be better to
store the cached data in another table?"Shabam" <blislecp@.hotmail.com> wrote in message
news:mcydnQLoHJrXdPvcRVn-tQ@.adelphia.com...
>I have an application that's allows user input, and is translating it by
> stripping out the html tags and also doing some code translations. The
> user
> is able to later edit their input. However it's unfeasible to reverse
> translate it back as the logic would be too complicated, and there are
> instances where it won't be possible.
> So, what I'm thinking to do to speed up performance is to duplicate the
> user
> data, one for native data, and the other for the translated data. When
> user
> edits their input, the native data is shown. When the application is
> showing the data in a page, the translated data is shown.
> My question is, would it make a performance difference if I store the
> native
> data and the translated data in the same table, or would it be better to
> store the cached data in another table?

It's not easy to say, without any information about the table structure,
data types, indexes, number of rows, query patterns etc. Splitting the table
is unlikely to have any impact by itself, unless perhaps you put the new
tables on different filegroups on different physical disks.

If you have performance issues, are you sure that I/O is the limiting
factor? Have you reviewed the query plans and used Profiler to look for
bottlenecks? You might also want to feed a Profiler trace into the Index
Tuning Wizard to see if it recommends an alternative indexing strategy.

Simon

Optimizing cursor performance?

Hi there!
I have an application that uses cursors en mass in SQL. Actually, it does not
speak one word relational SQL with MSSQL, only cursors. I think it has a fair
bet in becoming World Champion in cursor abuse.
Anyway, the application is almost pure sequential/flat file based, and thus I
wondered, is there anything I can do to MSSQL that pulls out every bit of cursor
performance available, as it is very much needed?
Microsoft knows the problems, as they themself bought this very application..
I doubt, therefore I might be.
Kim
Cursors are never (mostly) good choice. Without seeing your data ,it's hard
to suggest something.
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en mass in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>
|||I have found in many cases that declaring the cursor as Static can often
lead to increased performance but as always it depends<g>.
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en mass in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>
|||"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:Ov$xEZNvFHA.3864@.TK2MSFTNGP12.phx.gbl

> Cursors are never (mostly) good choice. Without seeing your data
> ,it's hard to suggest something.
I know, that's why I'm trying to be slightly sarcastic in my message

.
What do you need of information? Unfortunately I can only pull out limited
information on this, as the application is closed source.
I doubt, therefore I might be.
|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
> I have found in many cases that declaring the cursor as Static can
> often lead to increased performance but as always it depends<g>.
But is there anything more 'high level' I can do? What does cursors appreciate
of hardware; same as with relational SQL?
The fun thing is that the application doesn't seem to care much about extra RAM.
Even when the SQL server sucks up 1.5GB versus 500MB, I still don't see any
performance improvements noteworthy.
Almost seems like there's some hardwired cap on max cursor performance

.
I doubt, therefore I might be.
|||Depending on how they are configured and such they may use a lot of Tempdb.
So making sure tempdb is on a fast drive array might help. They tend to use
a fair amount of cpu as well. But there is no magic switch to make cursors
go faster. The real answer is sadly to rewrite them to not use cursors.
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:udfAjDSvFHA.2076@.TK2MSFTNGP14.phx.gbl...
> "Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
> news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
> But is there anything more 'high level' I can do? What does cursors
> appreciate of hardware; same as with relational SQL?
> The fun thing is that the application doesn't seem to care much about
> extra RAM. Even when the SQL server sucks up 1.5GB versus 500MB, I still
> don't see any performance improvements noteworthy.
> Almost seems like there's some hardwired cap on max cursor performance

.
> --
> I doubt, therefore I might be.
>
|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:e50F7YSvFHA.3580@.TK2MSFTNGP10.phx.gbl

> Depending on how they are configured and such they may use a lot of
> Tempdb. So making sure tempdb is on a fast drive array might help.
> They tend to use a fair amount of cpu as well. But there is no magic
> switch to make cursors go faster. The real answer is sadly to
> rewrite them to not use cursors.
I tried "performance monitor" the tempdb while pounding it with nasty cursors
through the application. However I don't seen any kind of activity in the
tempdb. Looking at the database the application uses, I get a "log byte
flushed/sec" activity (about 61kb) every 4-5 second or so.
Monitoring CPU usage I get around ~60% on a hyperthread Intel P4. "Monitoring
Avg. Disk Transfer" I get numbers far below the capacity of the disk (it's a WDC
Raptor 10k rpm).
So I'm not really any closer figuring out where the bottleneck is regarding
cursor abuse.
I doubt, therefore I might be.

Optimizing cursor performance?

Hi there!
I have an application that uses cursors en massé in SQL. Actually, it does not
speak one word relational SQL with MSSQL, only cursors. I think it has a fair
bet in becoming World Champion in cursor abuse.
Anyway, the application is almost pure sequential/flat file based, and thus I
wondered, is there anything I can do to MSSQL that pulls out every bit of cursor
performance available, as it is very much needed?
Microsoft knows the problems, as they themself bought this very application..
--
I doubt, therefore I might be.Kim
Cursors are never (mostly) good choice. Without seeing your data ,it's hard
to suggest something.
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en massé in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>|||I have found in many cases that declaring the cursor as Static can often
lead to increased performance but as always it depends<g>.
--
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en massé in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>|||"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:Ov$xEZNvFHA.3864@.TK2MSFTNGP12.phx.gbl
> Cursors are never (mostly) good choice. Without seeing your data
> ,it's hard to suggest something.
I know, that's why I'm trying to be slightly sarcastic in my message :).
What do you need of information? Unfortunately I can only pull out limited
information on this, as the application is closed source.
--
I doubt, therefore I might be.|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
> I have found in many cases that declaring the cursor as Static can
> often lead to increased performance but as always it depends<g>.
But is there anything more 'high level' I can do? What does cursors appreciate
of hardware; same as with relational SQL?
The fun thing is that the application doesn't seem to care much about extra RAM.
Even when the SQL server sucks up 1.5GB versus 500MB, I still don't see any
performance improvements noteworthy.
Almost seems like there's some hardwired cap on max cursor performance :).
--
I doubt, therefore I might be.|||Depending on how they are configured and such they may use a lot of Tempdb.
So making sure tempdb is on a fast drive array might help. They tend to use
a fair amount of cpu as well. But there is no magic switch to make cursors
go faster. The real answer is sadly to rewrite them to not use cursors.
--
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:udfAjDSvFHA.2076@.TK2MSFTNGP14.phx.gbl...
> "Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
> news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
>> I have found in many cases that declaring the cursor as Static can
>> often lead to increased performance but as always it depends<g>.
> But is there anything more 'high level' I can do? What does cursors
> appreciate of hardware; same as with relational SQL?
> The fun thing is that the application doesn't seem to care much about
> extra RAM. Even when the SQL server sucks up 1.5GB versus 500MB, I still
> don't see any performance improvements noteworthy.
> Almost seems like there's some hardwired cap on max cursor performance :).
> --
> I doubt, therefore I might be.
>|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:e50F7YSvFHA.3580@.TK2MSFTNGP10.phx.gbl
> Depending on how they are configured and such they may use a lot of
> Tempdb. So making sure tempdb is on a fast drive array might help.
> They tend to use a fair amount of cpu as well. But there is no magic
> switch to make cursors go faster. The real answer is sadly to
> rewrite them to not use cursors.
I tried "performance monitor" the tempdb while pounding it with nasty cursors
through the application. However I don't seen any kind of activity in the
tempdb. Looking at the database the application uses, I get a "log byte
flushed/sec" activity (about 61kb) every 4-5 second or so.
Monitoring CPU usage I get around ~60% on a hyperthread Intel P4. "Monitoring
Avg. Disk Transfer" I get numbers far below the capacity of the disk (it's a WDC
Raptor 10k rpm).
So I'm not really any closer figuring out where the bottleneck is regarding
cursor abuse.
--
I doubt, therefore I might be.

Optimizing cursor performance?

Hi there!
I have an application that uses cursors en mass in SQL. Actually, it does n
ot
speak one word relational SQL with MSSQL, only cursors. I think it has a fai
r
bet in becoming World Champion in cursor abuse.
Anyway, the application is almost pure sequential/flat file based, and thus
I
wondered, is there anything I can do to MSSQL that pulls out every bit of cu
rsor
performance available, as it is very much needed?
Microsoft knows the problems, as they themself bought this very application.
.
I doubt, therefore I might be.Kim
Cursors are never (mostly) good choice. Without seeing your data ,it's hard
to suggest something.
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en mass in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>|||I have found in many cases that declaring the cursor as Static can often
lead to increased performance but as always it depends<g>.
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:%23NlLrRMvFHA.2924@.TK2MSFTNGP15.phx.gbl...
> Hi there!
> I have an application that uses cursors en mass in SQL. Actually, it does
> not speak one word relational SQL with MSSQL, only cursors. I think it has
> a fair bet in becoming World Champion in cursor abuse.
> Anyway, the application is almost pure sequential/flat file based, and
> thus I wondered, is there anything I can do to MSSQL that pulls out every
> bit of cursor performance available, as it is very much needed?
> Microsoft knows the problems, as they themself bought this very
> application..
> --
> I doubt, therefore I might be.
>|||"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:Ov$xEZNvFHA.3864@.TK2MSFTNGP12.phx.gbl

> Cursors are never (mostly) good choice. Without seeing your data
> ,it's hard to suggest something.
I know, that's why I'm trying to be slightly sarcastic in my message

.
What do you need of information? Unfortunately I can only pull out limited
information on this, as the application is closed source.
I doubt, therefore I might be.|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
> I have found in many cases that declaring the cursor as Static can
> often lead to increased performance but as always it depends<g>.
But is there anything more 'high level' I can do? What does cursors apprecia
te
of hardware; same as with relational SQL?
The fun thing is that the application doesn't seem to care much about extra
RAM.
Even when the SQL server sucks up 1.5GB versus 500MB, I still don't see any
performance improvements noteworthy.
Almost seems like there's some hardwired cap on max cursor performance

.
I doubt, therefore I might be.|||Depending on how they are configured and such they may use a lot of Tempdb.
So making sure tempdb is on a fast drive array might help. They tend to use
a fair amount of cpu as well. But there is no magic switch to make cursors
go faster. The real answer is sadly to rewrite them to not use cursors.
Andrew J. Kelly SQL MVP
"Kim Noer" <kn@.nospam.dk> wrote in message
news:udfAjDSvFHA.2076@.TK2MSFTNGP14.phx.gbl...
> "Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
> news:%23$zCDeRvFHA.3588@.tk2msftngp13.phx.gbl
> But is there anything more 'high level' I can do? What does cursors
> appreciate of hardware; same as with relational SQL?
> The fun thing is that the application doesn't seem to care much about
> extra RAM. Even when the SQL server sucks up 1.5GB versus 500MB, I still
> don't see any performance improvements noteworthy.
> Almost seems like there's some hardwired cap on max cursor performance

.
> --
> I doubt, therefore I might be.
>|||"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:e50F7YSvFHA.3580@.TK2MSFTNGP10.phx.gbl

> Depending on how they are configured and such they may use a lot of
> Tempdb. So making sure tempdb is on a fast drive array might help.
> They tend to use a fair amount of cpu as well. But there is no magic
> switch to make cursors go faster. The real answer is sadly to
> rewrite them to not use cursors.
I tried "performance monitor" the tempdb while pounding it with nasty cursor
s
through the application. However I don't seen any kind of activity in the
tempdb. Looking at the database the application uses, I get a "log byte
flushed/sec" activity (about 61kb) every 4-5 second or so.
Monitoring CPU usage I get around ~60% on a hyperthread Intel P4. "Monitorin
g
Avg. Disk Transfer" I get numbers far below the capacity of the disk (it's a
WDC
Raptor 10k rpm).
So I'm not really any closer figuring out where the bottleneck is regarding
cursor abuse.
I doubt, therefore I might be.

Monday, March 26, 2012

optimized name-address-zipcode-region design

Whether thin-client with round-trip cycles to the db or a rich-client
loading look-up data into memory on application startup, what is a solid
optimized database design for customer location search functionality. USPS
zip data has about 75K records.
Any articles on this? Thanks. -Greg
**** ** Application supplied data *******
1. First Name (PrimaryContactID)
2. Last Name
3. Client Name
4. Address (AddressID)
****** Vendor supplied data
zipinfo.com,/www.zip-codes.com,www.buyzips.com,etc. *******
5. City (CityID)
6. Region (State,Province, etc) (RegionID)
7. Postal Code. (PostalCodeID)
8. Area Code (AreaCodeID)
an optimized search that would return all customers (within a second) for
any of the following:
1. a given zipcode/postalcode
2. a given State/Region,
3. a given City
4. a given Area Code
5. a given match on name or substring of address."hazz" <hazz@.sonic_net> wrote in
news:ejBkMi7bGHA.3908@.TK2MSFTNGP02.phx.gbl:

> Whether thin-client with round-trip cycles to the db or a rich-client
> loading look-up data into memory on application startup, what is a
> solid optimized database design for customer location search
> functionality. USPS zip data has about 75K records.
> Any articles on this? Thanks. -Greg
> **** ** Application supplied data *******
> 1. First Name (PrimaryContactID)
> 2. Last Name
> 3. Client Name
> 4. Address (AddressID)
> ****** Vendor supplied data
> zipinfo.com,/www.zip-codes.com,www.buyzips.com,etc. *******
> 5. City (CityID)
> 6. Region (State,Province, etc) (RegionID)
> 7. Postal Code. (PostalCodeID)
> 8. Area Code (AreaCodeID)
> an optimized search that would return all customers (within a second)
> for any of the following:
> 1. a given zipcode/postalcode
> 2. a given State/Region,
> 3. a given City
> 4. a given Area Code
> 5. a given match on name or substring of address.
>
>
Here is a link to an article I read this w

on ASP FAQ
How do I create a store locator feature?
http://www.aspfaq.com/show.asp?id=2527
- Jose|||Thank you for that link Jose. That is not exactly what I was after but that
is interesting. The vendor db's do have latitude and longitude so this is a
good example of how that info is useful. -Greg
"Jose Hernandez" <acmp_dev@.yahoo.com> wrote in message
news:Xns97B98A2952D76acmpdevyahoocom@.207
.46.248.16...
> "hazz" <hazz@.sonic_net> wrote in
> news:ejBkMi7bGHA.3908@.TK2MSFTNGP02.phx.gbl:
>
> Here is a link to an article I read this w

on ASP FAQ
> How do I create a store locator feature?
> http://www.aspfaq.com/show.asp?id=2527
>
> - Jose|||You can also get the postal code database from several vendors such as
http://www.zipcodeworld.com.

Optimize Temp tables

Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,
> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
With that much code running just to generate a report, you might consider
having a background process (or a data warehouse) pre-aggregate the data.
This is my signature. It is a general reminder.
Please post DDL, sample data and desired results.
See http://www.aspfaq.com/5006 for info.
|||In addition to Aaron's response , create a temp tables at the beginning of
the stored procedure (reduce recompile).
Also, it may help to improve performance if you add NC/CI indexes depending
on your requrements.
Look at an execution plan of the stored procedure to ensure that optimizer
is available to use indexes.
"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegr oups.com...
> Hi All,
> My application makes heavy use of temp tables because we have a few
> thousands of stored proc running every day to renerate reports.
> What are the positive and negative impact to add clustered index on the
> temp table?
> what is the negative impact if I use derived table instead temp table?
> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
> Thanks so much,
> Silaphet,
>
|||Another possibility would be to consider an INDEXED VIEW.
Typically, you want to keep your OLTP and DSS/OLAP operations and data
seperated. It might be wise to reconsider the cohosting of this data in the
same database and reconsider creating something new elsewhere that better
fits your architecture.
Sincerely,
Anthony Thomas

"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegr oups.com...
Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,

Optimize Temp tables

Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
With that much code running just to generate a report, you might consider
having a background process (or a data warehouse) pre-aggregate the data.
This is my signature. It is a general reminder.
Please post DDL, sample data and desired results.
See http://www.aspfaq.com/5006 for info.|||In addition to Aaron's response , create a temp tables at the beginning of
the stored procedure (reduce recompile).
Also, it may help to improve performance if you add NC/CI indexes depending
on your requrements.
Look at an execution plan of the stored procedure to ensure that optimizer
is available to use indexes.
"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegroups.com...
> Hi All,
> My application makes heavy use of temp tables because we have a few
> thousands of stored proc running every day to renerate reports.
> What are the positive and negative impact to add clustered index on the
> temp table?
> what is the negative impact if I use derived table instead temp table?
> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
> Thanks so much,
> Silaphet,
>|||Another possibility would be to consider an INDEXED VIEW.
Typically, you want to keep your OLTP and DSS/OLAP operations and data
seperated. It might be wise to reconsider the cohosting of this data in the
same database and reconsider creating something new elsewhere that better
fits your architecture.
Sincerely,
Anthony Thomas
"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegroups.com...
Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,

Optimize Temp tables

Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
With that much code running just to generate a report, you might consider
having a background process (or a data warehouse) pre-aggregate the data.
--
This is my signature. It is a general reminder.
Please post DDL, sample data and desired results.
See http://www.aspfaq.com/5006 for info.|||In addition to Aaron's response , create a temp tables at the beginning of
the stored procedure (reduce recompile).
Also, it may help to improve performance if you add NC/CI indexes depending
on your requrements.
Look at an execution plan of the stored procedure to ensure that optimizer
is available to use indexes.
"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegroups.com...
> Hi All,
> My application makes heavy use of temp tables because we have a few
> thousands of stored proc running every day to renerate reports.
> What are the positive and negative impact to add clustered index on the
> temp table?
> what is the negative impact if I use derived table instead temp table?
> When I have big stored proc more than 1500 lines and have at least 5
> temp tables, what is the best solution to make the heavy stored proc
> run faster?
> Thanks so much,
> Silaphet,
>|||Another possibility would be to consider an INDEXED VIEW.
Typically, you want to keep your OLTP and DSS/OLAP operations and data
seperated. It might be wise to reconsider the cohosting of this data in the
same database and reconsider creating something new elsewhere that better
fits your architecture.
Sincerely,
Anthony Thomas
"Silaphet" <kmounkhaty@.cox.net> wrote in message
news:1112583294.022932.260500@.z14g2000cwz.googlegroups.com...
Hi All,
My application makes heavy use of temp tables because we have a few
thousands of stored proc running every day to renerate reports.
What are the positive and negative impact to add clustered index on the
temp table?
what is the negative impact if I use derived table instead temp table?
When I have big stored proc more than 1500 lines and have at least 5
temp tables, what is the best solution to make the heavy stored proc
run faster?
Thanks so much,
Silaphet,

Friday, March 23, 2012

optimizations job for db maintenance plan failed

This is happening on two of our servers.
We get the warning in the application log as seen here:
http://support.microsoft.com/kb/902388/
But we don't get the SQL Server log entry that is mentioned in that KB
article.
Here are the commands from the jobs (after adding the
option -SupportComputedColumn , as recommended in the KB article) :
Server 1:
EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
10 -SupportComputedColumn '
Server 2:
EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
"G:\MSSQL\MSSQL\LOG\User DB Maintenance0.txt" -WriteHistory -UpdOptiStats
10 -SupportComputedColumn '
Any suggestions, anyone?
Regards,
JimDo you have the right service pack to support that?
--
Andrew J. Kelly SQL MVP
"Jim Moon" <please.reply@.group> wrote in message
news:%23TYZbMLXGHA.1564@.TK2MSFTNGP03.phx.gbl...
> This is happening on two of our servers.
> We get the warning in the application log as seen here:
> http://support.microsoft.com/kb/902388/
> But we don't get the SQL Server log entry that is mentioned in that KB
> article.
> Here are the commands from the jobs (after adding the
> option -SupportComputedColumn , as recommended in the KB article) :
> Server 1:
> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
> 10 -SupportComputedColumn '
> Server 2:
> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
> "G:\MSSQL\MSSQL\LOG\User DB Maintenance0.txt" -WriteHistory -UpdOptiStats
> 10 -SupportComputedColumn '
> Any suggestions, anyone?
> Regards,
> Jim
>|||Does anyone at all have any ideas?
I would be very grateful for any relevant feedback! :-)
Thanks,
Jim
"Jim Moon" <please.reply@.group> wrote in message
news:%23TYZbMLXGHA.1564@.TK2MSFTNGP03.phx.gbl...
> This is happening on two of our servers.
> We get the warning in the application log as seen here:
> http://support.microsoft.com/kb/902388/
> But we don't get the SQL Server log entry that is mentioned in that KB
> article.
> Here are the commands from the jobs (after adding the
> option -SupportComputedColumn , as recommended in the KB article) :
> Server 1:
> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
> 10 -SupportComputedColumn '
> Server 2:
> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
> "G:\MSSQL\MSSQL\LOG\User DB Maintenance0.txt" -WriteHistory -UpdOptiStats
> 10 -SupportComputedColumn '
> Any suggestions, anyone?
> Regards,
> Jim
>|||Why would that not be relevant? This feature was not a part of SQL Server
until Service Pack 4. Since you did not post what version you are using it
is quite feasible that it is not working because that service pack does not
support it.
--
Andrew J. Kelly SQL MVP
"Jim Moon" <please.reply@.group> wrote in message
news:%23KBVO%23xXGHA.3496@.TK2MSFTNGP05.phx.gbl...
> Does anyone at all have any ideas?
> I would be very grateful for any relevant feedback! :-)
> Thanks,
> Jim
> "Jim Moon" <please.reply@.group> wrote in message
> news:%23TYZbMLXGHA.1564@.TK2MSFTNGP03.phx.gbl...
>> This is happening on two of our servers.
>> We get the warning in the application log as seen here:
>> http://support.microsoft.com/kb/902388/
>> But we don't get the SQL Server log entry that is mentioned in that KB
>> article.
>> Here are the commands from the jobs (after adding the
>> option -SupportComputedColumn , as recommended in the KB article) :
>> Server 1:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
>> 10 -SupportComputedColumn '
>> Server 2:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
>> "G:\MSSQL\MSSQL\LOG\User DB
>> aintenance0.txt" -WriteHistory -UpdOptiStats 10 -SupportComputedColumn '
>> Any suggestions, anyone?
>> Regards,
>> Jim
>|||Hi Andrew,
It is SQL Server SP4.
Regards,
Jim
"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:OElKDt1XGHA.4476@.TK2MSFTNGP05.phx.gbl...
> Why would that not be relevant? This feature was not a part of SQL Server
> until Service Pack 4. Since you did not post what version you are using
> it is quite feasible that it is not working because that service pack does
> not support it.
> --
> Andrew J. Kelly SQL MVP
>
> "Jim Moon" <please.reply@.group> wrote in message
> news:%23KBVO%23xXGHA.3496@.TK2MSFTNGP05.phx.gbl...
>> Does anyone at all have any ideas?
>> I would be very grateful for any relevant feedback! :-)
>> Thanks,
>> Jim
>> "Jim Moon" <please.reply@.group> wrote in message
>> news:%23TYZbMLXGHA.1564@.TK2MSFTNGP03.phx.gbl...
>> This is happening on two of our servers.
>> We get the warning in the application log as seen here:
>> http://support.microsoft.com/kb/902388/
>> But we don't get the SQL Server log entry that is mentioned in that KB
>> article.
>> Here are the commands from the jobs (after adding the
>> option -SupportComputedColumn , as recommended in the KB article) :
>> Server 1:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
>> 10 -SupportComputedColumn '
>> Server 2:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
>> "G:\MSSQL\MSSQL\LOG\User DB
>> intenance0.txt" -WriteHistory -UpdOptiStats 10 -SupportComputedColumn '
>> Any suggestions, anyone?
>> Regards,
>> Jim
>>
>|||Jim,
SP4 and that option was supposed to fix those issues and I haven't heard of
anyone else using them and it not working as expected. Unless someone does
pipe in with something else you really have two options. One is to call MS
PSS and report this. If it is a bug they will refund your initial trouble
ticket charge. If not it will cost about $225. The other and recommended
option is to forget about the Maintenance plans either altogether or at
least for this part. It looks like you are having issues with the updating
of the stats. First I would ask why you are using the MP for that? Do you
also reindex each table either with DBCC DBREINDEX or with the maintenance
plans? If so then that will update the stats for you and this step is
unnecessary. Otherwise you can create your own job and simply run
sp_updatestats and be done with it.
--
Andrew J. Kelly SQL MVP
"Jim Moon" <please.reply@.group> wrote in message
news:uScFlH%23XGHA.508@.TK2MSFTNGP02.phx.gbl...
> Hi Andrew,
> It is SQL Server SP4.
> Regards,
> Jim
>
> "Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
> news:OElKDt1XGHA.4476@.TK2MSFTNGP05.phx.gbl...
>> Why would that not be relevant? This feature was not a part of SQL
>> Server until Service Pack 4. Since you did not post what version you are
>> using it is quite feasible that it is not working because that service
>> pack does not support it.
>> --
>> Andrew J. Kelly SQL MVP
>>
>> "Jim Moon" <please.reply@.group> wrote in message
>> news:%23KBVO%23xXGHA.3496@.TK2MSFTNGP05.phx.gbl...
>> Does anyone at all have any ideas?
>> I would be very grateful for any relevant feedback! :-)
>> Thanks,
>> Jim
>> "Jim Moon" <please.reply@.group> wrote in message
>> news:%23TYZbMLXGHA.1564@.TK2MSFTNGP03.phx.gbl...
>> This is happening on two of our servers.
>> We get the warning in the application log as seen here:
>> http://support.microsoft.com/kb/902388/
>> But we don't get the SQL Server log entry that is mentioned in that KB
>> article.
>> Here are the commands from the jobs (after adding the
>> option -SupportComputedColumn , as recommended in the KB article) :
>> Server 1:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -UpdOptiStats
>> 10 -SupportComputedColumn '
>> Server 2:
>> EXECUTE master.dbo.xp_sqlmaint N'-PlanID <GUID> -Rpt
>> "G:\MSSQL\MSSQL\LOG\User DB
>> ntenance0.txt" -WriteHistory -UpdOptiStats 10 -SupportComputedColumn '
>> Any suggestions, anyone?
>> Regards,
>> Jim
>>
>>
>

optimizations job for db maintenance plan failed

Wednesday, March 21, 2012

Optimization issue

hi All,
I have a 2.5 GIG SQL2K SP3 database with an Access XP
front end application. As the database grew, the responce
time has degraded dramatically. It is now taking minutes
to sell a service, when it should only take seconds. all
together I have 20 different databases all using the same
front end. they do not seem to be having this issue.
their DB's are closer to 700 Megs.
I ran a DBCC CheckDb. there were no errors found. Are
there any ways to optimize the database? Would update
statistics, and sp_recompile help?run profiler with the default template to find the slow
query or procedure,
then paste that procedure into query analyzer
and look at the execution plan for that query
>--Original Message--
>hi All,
>I have a 2.5 GIG SQL2K SP3 database with an Access XP
>front end application. As the database grew, the
responce
>time has degraded dramatically. It is now taking minutes
>to sell a service, when it should only take seconds. all
>together I have 20 different databases all using the same
>front end. they do not seem to be having this issue.
>their DB's are closer to 700 Megs.
>I ran a DBCC CheckDb. there were no errors found. Are
>there any ways to optimize the database? Would update
>statistics, and sp_recompile help?
>
>.
>

Tuesday, March 20, 2012

Optimization

We're building a company wide network monitoring system
in Java, and need some advice on the database design and
tuning.

The application will need to concurrently INSERT,
DELETE, and SELECT from our EVENT table as efficiently as
possible. We plan to implement an INSERT thread, a DELETE
thread, and a SELECT thread within our Java program.

The EVENT table will have several hundred million records
in it at any given time. We will prune, using DELETE, about
every five seconds to keep the active record set down to
a user controlled size. And one of the three queries will
be executed about every twenty seconds. Finally, we'll
INSERT as fast as we can in the INSERT thread.

Being new to MSSQL, we need advice on

1) Server Tuning - Memory allocations, etc.
2) Table Tuning - Field types
3) Index Tuning - Are the indexes right
4) Query Tuning - Hints, etc.
5) Process Tuning - Better ways to INSERT and DELETE, etc.

Thanks, in advance, for any suggestions you can make :-)

The table is

// CREATE TABLE EVENT (
// ID INT PRIMARY KEY NOT NULL,
// IPSOURCE INT NOT NULL,
// IPDEST INT NOT NULL,
// UNIXTIME BIGINT NOT NULL,
// TYPE TINYINT NOT NULL,
// DEVICEID SMALLINT NOT NULL,
// PROTOCOL TINYINT NOT NULL
// )
//
// CREATE INDEX INDEX_SRC_DEST_TYPE
// ON EVENT (
// IPSOURCE,IPDEST,TYPE
// )

The SELECTS are

private static String QueryString1 =
"SELECT ID,IPSOURCE,IPDEST,TYPE "+
"FROM EVENT "+
"WHERE ID >= ? "+
" AND ID <= ?";

private static String QueryString2 =
"SELECT COUNT(*),IPSOURCE "+
"FROM EVENT "+
"GROUP BY IPSOURCE "+
"ORDER BY 1 DESC";

private static String QueryString3 =
"SELECT COUNT(*),IPDEST "+
"FROM EVENT "+
"WHERE IPSOURCE = ? "+
" AND TYPE = ? "+
"GROUP BY IPDEST "+
"ORDER BY 1 DESC";

The DELETE is

private static String DeleteIDString =
"DELETE FROM EVENT "+
"WHERE ID < ?";Hi,

Refer www.SQL-Server-Performance.com web site for SQL Server Performance issues.
Almost all yours queries id answered in perfect articles and example wise.

Thanks, Amit

pwhittington@.nitrodata.com (rocky) wrote in message news:<3961374c.0307041440.4ebb73ef@.posting.google.com>...
> We're building a company wide network monitoring system
> in Java, and need some advice on the database design and
> tuning.
> The application will need to concurrently INSERT,
> DELETE, and SELECT from our EVENT table as efficiently as
> possible. We plan to implement an INSERT thread, a DELETE
> thread, and a SELECT thread within our Java program.
> The EVENT table will have several hundred million records
> in it at any given time. We will prune, using DELETE, about
> every five seconds to keep the active record set down to
> a user controlled size. And one of the three queries will
> be executed about every twenty seconds. Finally, we'll
> INSERT as fast as we can in the INSERT thread.
> Being new to MSSQL, we need advice on
> 1) Server Tuning - Memory allocations, etc.
> 2) Table Tuning - Field types
> 3) Index Tuning - Are the indexes right
> 4) Query Tuning - Hints, etc.
> 5) Process Tuning - Better ways to INSERT and DELETE, etc.
> Thanks, in advance, for any suggestions you can make :-)
>
> The table is
> // CREATE TABLE EVENT (
> // ID INT PRIMARY KEY NOT NULL,
> // IPSOURCE INT NOT NULL,
> // IPDEST INT NOT NULL,
> // UNIXTIME BIGINT NOT NULL,
> // TYPE TINYINT NOT NULL,
> // DEVICEID SMALLINT NOT NULL,
> // PROTOCOL TINYINT NOT NULL
> // )
> //
> // CREATE INDEX INDEX_SRC_DEST_TYPE
> // ON EVENT (
> // IPSOURCE,IPDEST,TYPE
> // )
> The SELECTS are
> private static String QueryString1 =
> "SELECT ID,IPSOURCE,IPDEST,TYPE "+
> "FROM EVENT "+
> "WHERE ID >= ? "+
> " AND ID <= ?";
> private static String QueryString2 =
> "SELECT COUNT(*),IPSOURCE "+
> "FROM EVENT "+
> "GROUP BY IPSOURCE "+
> "ORDER BY 1 DESC";
> private static String QueryString3 =
> "SELECT COUNT(*),IPDEST "+
> "FROM EVENT "+
> "WHERE IPSOURCE = ? "+
> " AND TYPE = ? "+
> "GROUP BY IPDEST "+
> "ORDER BY 1 DESC";
> The DELETE is
> private static String DeleteIDString =
> "DELETE FROM EVENT "+
> "WHERE ID < ?";|||Some mixed advice:

1) Make sure that you have plenty of memory in the server. Query2 will
always generate a full index scan, and you need to be able to fit all of the
data for that index in memory to avoid excessive disk I/O. 100 million rows
* 13 bytes = 1.3 GB of memory. I would suggest having at least 2GB memory in
the server.

2) Make sure you have a disk subsystem with good performance for both
reading and writing. Use several striped disks for the data (not RAID5), and
a separate disk for log. Use a disk controller with a battery-backed write
cache. The disk performance is important even if you have plenty of memory,
but it is absolutely vital if you can not fit all the data in memory.

3) Add a NOLOCK hint to all SELECT queries. Otherwise, all inserts will be
blocked while executing the SELECTs.

4) Change the index to IPSOURCE,TYPE,IPDEST to optimize Query3

5) Make sure that you batch inserts. Have the INSERT thread generate batches
of inserts and send them all to the server in a single batch as a single
transaction. Having a transaction for each inserted row will kill
performance unless you have a really good disk controller.

/SG

"rocky" <pwhittington@.nitrodata.com> wrote in message
news:3961374c.0307041440.4ebb73ef@.posting.google.c om...
> We're building a company wide network monitoring system
> in Java, and need some advice on the database design and
> tuning.
> The application will need to concurrently INSERT,
> DELETE, and SELECT from our EVENT table as efficiently as
> possible. We plan to implement an INSERT thread, a DELETE
> thread, and a SELECT thread within our Java program.
> The EVENT table will have several hundred million records
> in it at any given time. We will prune, using DELETE, about
> every five seconds to keep the active record set down to
> a user controlled size. And one of the three queries will
> be executed about every twenty seconds. Finally, we'll
> INSERT as fast as we can in the INSERT thread.
> Being new to MSSQL, we need advice on
> 1) Server Tuning - Memory allocations, etc.
> 2) Table Tuning - Field types
> 3) Index Tuning - Are the indexes right
> 4) Query Tuning - Hints, etc.
> 5) Process Tuning - Better ways to INSERT and DELETE, etc.
> Thanks, in advance, for any suggestions you can make :-)
>
> The table is
> // CREATE TABLE EVENT (
> // ID INT PRIMARY KEY NOT NULL,
> // IPSOURCE INT NOT NULL,
> // IPDEST INT NOT NULL,
> // UNIXTIME BIGINT NOT NULL,
> // TYPE TINYINT NOT NULL,
> // DEVICEID SMALLINT NOT NULL,
> // PROTOCOL TINYINT NOT NULL
> // )
> //
> // CREATE INDEX INDEX_SRC_DEST_TYPE
> // ON EVENT (
> // IPSOURCE,IPDEST,TYPE
> // )
> The SELECTS are
> private static String QueryString1 =
> "SELECT ID,IPSOURCE,IPDEST,TYPE "+
> "FROM EVENT "+
> "WHERE ID >= ? "+
> " AND ID <= ?";
> private static String QueryString2 =
> "SELECT COUNT(*),IPSOURCE "+
> "FROM EVENT "+
> "GROUP BY IPSOURCE "+
> "ORDER BY 1 DESC";
> private static String QueryString3 =
> "SELECT COUNT(*),IPDEST "+
> "FROM EVENT "+
> "WHERE IPSOURCE = ? "+
> " AND TYPE = ? "+
> "GROUP BY IPDEST "+
> "ORDER BY 1 DESC";
> The DELETE is
> private static String DeleteIDString =
> "DELETE FROM EVENT "+
> "WHERE ID < ?";

optimistic locking

I am writing an Application which is connected to a database on MSSQl
Server. For performing updates on a table I chose the way of optimistic
locking using timestamps. I use a stored procedure for updating the
values which returns either 0 or 1 to indicate if there was a conflict
or not. If there was one, the current values in the database are selected.
My stored procedure for updating data looks like this:
create procedure updateSomeTable
@.key int,
@.col1 nvarchar(100),
@.col2 datetime,
@.TS timestamp
AS
update MyDB.dbo.SomeTable
set col1 = @.col1,
col2 = @.col2
where key = @.key and TS = @.TS
if(@.@.rowcount = 0)
begin
select * from SomeTable where key = @.key
return 1
end
return 0
Do I need any transactions within souch a procedure? As far as I can
think of, I don't need any... Is this the typical way to perform this
task, or are there much more common ways to do this?
Thanks
Regards
StephanNot really. The UPDATE command is in an implicit transaction, and it is the
nature of the beast of opimistic locking that a user might change it after
you retrieve it (it is optimistic after all.)
Not sure that I would just conditionally receive the row. The return count
here is not of much value, since you already have to decide whether or not
to fetch a dataset which may or may not be returned. I would just always
return the row, which takes care of any trigger modifications (if there were
any, usually this in not a big deal) but also to handle the new timestamp.
Once you do the update it will change, meaning you will have to go fetch at
least the timestamp after the modification.
----
Louis Davidson - http://spaces.msn.com/members/drsql/
SQL Server MVP
"Arguments are to be avoided: they are always vulgar and often convincing."
(Oscar Wilde)
"Stephan Zaubzer" <stephan.zaubzer@.schendl.at> wrote in message
news:utkvIuQIGHA.2300@.TK2MSFTNGP15.phx.gbl...
>I am writing an Application which is connected to a database on MSSQl
>Server. For performing updates on a table I chose the way of optimistic
>locking using timestamps. I use a stored procedure for updating the values
>which returns either 0 or 1 to indicate if there was a conflict or not. If
>there was one, the current values in the database are selected.
> My stored procedure for updating data looks like this:
> create procedure updateSomeTable
> @.key int,
> @.col1 nvarchar(100),
> @.col2 datetime,
> @.TS timestamp
> AS
> update MyDB.dbo.SomeTable
> set col1 = @.col1,
> col2 = @.col2
> where key = @.key and TS = @.TS
> if(@.@.rowcount = 0)
> begin
> select * from SomeTable where key = @.key
> return 1
> end
> return 0
> Do I need any transactions within souch a procedure? As far as I can think
> of, I don't need any... Is this the typical way to perform this task, or
> are there much more common ways to do this?
> Thanks
> Regards
> Stephan|||Ok, so my thoughts of not to need an explicit transaction was ok...
As you said it is definitely better to fetch the new dataset immediately
after inserting it. But the chances exist, that someone will update this
row between my insert and the fetch. But this probability should rather
be negligible.
Are there any other issues besides these with this procedure?
Louis Davidson schrieb:
> Not really. The UPDATE command is in an implicit transaction, and it is t
he
> nature of the beast of opimistic locking that a user might change it after
> you retrieve it (it is optimistic after all.)
> Not sure that I would just conditionally receive the row. The return coun
t
> here is not of much value, since you already have to decide whether or not
> to fetch a dataset which may or may not be returned. I would just always
> return the row, which takes care of any trigger modifications (if there we
re
> any, usually this in not a big deal) but also to handle the new timestamp.
> Once you do the update it will change, meaning you will have to go fetch a
t
> least the timestamp after the modification.
>|||> after inserting it. But the chances exist, that someone will update this
> row between my insert and the fetch. But this probability should rather be
> negligible.
This really isn't a big deal at all. The idea behind optimistic locking is
that you can get the rows, hold onto them for a long time, and still know
that they are good by the timestamp. You just accept this as the risk
(obviously the risk should be evaluated based on the data integrity
requirements).
I am going to assume that this is for 2000. If it is 2005, we could extend
using TRY...CATCH. You would possibly want transactions for 2005 with
TRY...CATCH because of how triggers behave. In 2000 an error/rollback in a
trigger just halted everything. If you are in a TRY...CATCH block, that
changes.
Anyhow, I wouldn't put the database name as a prefix, unless it is in a
different one from the tables. It makes it harder to use it with a
different named database, like for testing if you don't have the ability to
have multiple servers. I think this will work fine. I added a check to see
if the row had been deleted, though you will also know that because no rows
will be returned :)
create procedure updateSomeTable
@.key int,
@.col1 nvarchar(100),
@.col2 datetime,
@.TS timestamp
AS
declare @.rowcount int
update dbo.SomeTable
set col1 = @.col1,
col2 = @.col2
where key = @.key
and TS = @.TS
set @.rowcount = @.@.rowcount
select col1, col2
from dbo.SomeTable
where key= @.key
--this means that the row has been deleted
if @.rowcount = 1 --this means it has been updated
return 1
else
return 0
go
----
Louis Davidson - http://spaces.msn.com/members/drsql/
SQL Server MVP
"Arguments are to be avoided: they are always vulgar and often convincing."
(Oscar Wilde)
"Stephan Zaubzer" <stephan.zaubzer@.schendl.at> wrote in message
news:%235J5fSSIGHA.2696@.TK2MSFTNGP14.phx.gbl...
> Ok, so my thoughts of not to need an explicit transaction was ok...
> As you said it is definitely better to fetch the new dataset immediately
> after inserting it. But the chances exist, that someone will update this
> row between my insert and the fetch. But this probability should rather be
> negligible.
> Are there any other issues besides these with this procedure?
> Louis Davidson schrieb:

Monday, March 19, 2012

Optimising Select statements which has a LIKE where clause.

Hi all

I have been doing some development work in a large VB6 application. I have updated the search capabilities of the application to allow the user to search on partial addresses as the existing search routine only allowed you to search on the whole line of the address.

Simple change to the stored procedure (this is just an example not the real stored proc):

From:
Select Top 3000 * from TL_ClientAddresses with(nolock) Where strPostCode = W1 ABC
To:
Select Top 3000 * from TL_ClientAddresses with(nolock) Where strPostCode LIKE W1%

Now this is when things went a bit crazy. I know the implications of using with(nolock). But seeing the code is only using the ID field to get the required row, and the database is a live database with hundreds of users at any one time (some updating), I think a dirty read is ok in this routine, as I dont want SQL to create a shared lock.

Anyway my problem is this. After the change, the search now created a Shared Lock which sometimes locks out some of the live users updating the system. The Select is also extremely SLOW. It took about 5 minutes to search just over a million records (locking the database during the search, and giving my manager good reason to shout abuse at me). So I checked the indexes. I had an index set on:

strAddressLine1, strAddressLine2, strAddressLine3, strAddressLine4, strPostCode.

So I created an index just for the strPostCode (non clustered).

This had no change to the Like select what so ever. So I am now stuck.

1) Is there another way to search for part of a text field in SQL.
2) Does Like comparison use the index in any way? If so how do I set this index up?
3) Can I stop a Shared Lock being created when I do a like select?
4) Do you have any good comebacks I could tell the boss after his next outburst of abuse (please not so bad that he sacks me).

Any advice truly appreciated.1. I have been working on smaller database systems the last couple of years but as for number 1 try changing the query to "=" with a wild card "%" in the QA with the show execution plan on and see if the index is being used.

2. I seem to remember that the like operator cancels the index. To check this I would execute the query in the QA and check the execution plan.

3. Don't know off the top of my head.

4. Tell your boss that software like the people who create it are imperfect things.|||Like is one of those "fuzzy" things and does not use indexes. Postcodes are notoriously difficult to search on...

In your example you give "W1%"

realise that this will return W12 etc

I in the time I had to do this tried to Narrow the user down to the local as in W1
W12 etc

Alternative prospect here

Split your Postcode into two fields (I know it sounds wierd) but then you can do an = rather than a like and an index can be used!|||I'd try to run the query in the Query Analyzer. In the form that you posted the query, it ought to use the PostCode index. It ought to be able to ride the index as far as the first wildcard (percent sign in this case). If you examine the query plan, it might give you some idea of where the problem is.

-PatP|||I get an index seek with a bookmark lookup

USE Northwind
GO

SET NOCOUNT ON
CREATE TABLE myTable99(strPostCode varchar(10), Col2 int)
GO

INSERT INTO myTable99(strPostCode, Col2)
SELECT 'W1Me',1 UNION ALL
SELECT 'W2Me',2 UNION ALL
SELECT 'W3Me',3 UNION ALL
SELECT 'W4Me',4
GO

CREATE INDEX myTable99_strPostCode ON myTable99(strPostCode)
GO

--[CTRL]+K
Select Top 3000 * from myTable99 with(nolock) Where strPostCode LIKE 'W1%'
GO

SET NOCOUNT OFF
GO|||Yeah, but that's using the code that they posted, which might or might not generate the same plan as the code they are actually using ;) Not that I've ever been burned by different code being posted than what is actually run before, I just read about it in this book once...

-PatP|||First like to thank everyone who replied. Very much appreciated.

Unfortunately I am still not much closer at finding a solution. I have done the execution plan thing although I can see a index seek, I dont think the index I have created is being used (but I am not sure).

If I delete the index, then the search time on about 500,000 records (Im testing from a subset of the total number of rows in the table) is about the same as when the index is present. If I do a = search, then with an index runs in less than a second and without takes much longer.

Anyway if anyone has any other ideas or a total different approach I can take then please let me know. Also if anyone knows of any websites or books that look into like selects more deeply than just giving you the syntax, like every book and site I have found, then that would be good too.

Regards

Eamon.|||1. Try use index hint:
Select Top 3000 * from TL_ClientAddresses with(nolock, INDEX (your_index_name)) Where strPostCode LIKE W1%
2. Try update statistics:
EXEC sp_updatestats
3. Try change query:
Select Top 3000 * from TL_ClientAddresses with(nolock)
Where strPostCode >= W1 AND strPostCode =< W1z
4. If your query fetch more then 20% table then optimizer don't use index|||mwolf! Your a Star!!!

Select Top 3000 * from TL_ClientAddresses with(nolock, INDEX (your_index_name)) Where strPostCode LIKE W1%

Works a treat!!! Gone from over a minute down to 6 seconds just by adding the 'INDEX()' statement.

I think that's the answer to my problem, Thankyou very much!!!

Regards

Eamon.

optimising a table with lots of boolean fields

I have an application that reads a monitoring devices that produces 200 digital outputs every second and I would like to store them in a table. This table would get quite big fairly quickly as ultimately I would like to monitor over a hundred of these devices.

I would like to construct queries against each of the individual digital channels or combinations of them.

M first thought is to set up a table with 200 separate columns (plus others for date stamp, device ID etc) however, I am concerned that a table with 200 boolean (1-bit) fields would be an enormous waste of space if each field takes maybe one to four bytes on the hard disk to store a single bit. However, this would have the advantage of make the SQL queries more natural.

The other alternative is to create a single 200 bit field and use lots of ANDing and ORing to isolate bits to do my queries. This would make my SQL code less readable and may also cause nore hassle in the future if the inputs changed, but it would make the file size smaller.

In essence I am asking (hoping) the following : If I create a table with 200 boolean fields, does SQL server express automatically optimise the storage to make it more compact? This means that the server can mess around at the bit level and leave my higher level SQL code looking cleaner and more logical.

hi,

SQL Server can pad up to 8 bit columns together to save space, but this is another concern in your problem..

I'd actually go for a more normalized model.. you can break the 200 bit columns in a separated entity referincing the same transaction... this not only is more "elegant" and correct, but solves problem where you have to add/modify a device..

something like

SET NOCOUNT ON;

USE tempdb;

CREATE TABLE dbo.Devices (

Id int NOT NULL PRIMARY KEY,

Description varchar(10) NOT NULL

);

CREATE TABLE dbo.DeviceTran (

Id int NOT NULL IDENTITY PRIMARY KEY , -- for sake of simplicity

otherData varchar(10) NULL,

TimeRecorded datetime DEFAULT GETDATE()

);

CREATE TABLE dbo.DeviceTranOutput (

Id int NOT NULL IDENTITY PRIMARY KEY , -- for sake of simplicity

IdTran int NOT NULL

CONSTRAINT fk_DeviceTran_DeviceTranOutput

FOREIGN KEY

REFERENCES dbo.DeviceTran (Id),

IdDevice int NOT NULL

CONSTRAINT fk_Devices_DeviceTranOutput

FOREIGN KEY

REFERENCES dbo.Devices (Id),

TValue bit NOT NULL DEFAULT 0

);

PRINT 'available devices';

INSERT INTO dbo.Devices VALUES ( 1 , 'PDA' );

INSERT INTO dbo.Devices VALUES ( 2 , 'PBAX' );

INSERT INTO dbo.Devices VALUES ( 3 , 'PC' );

INSERT INTO dbo.Devices VALUES ( 4 , 'xxx' );

SELECT * FROM dbo.Devices;

PRINT '--';

DECLARE @.i int, @.y int, @.id int;

SET @.i = 1

WHILE @.i < 10 BEGIN

INSERT INTO dbo.DeviceTran VALUES ( 'Data ' + CONVERT(varchar, @.i), DEFAULT);

SELECT @.id = SCOPE_IDENTITY();

SET @.y = 1;

WHILE @.y < 5 BEGIN

INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.y , (@.i + @.y) % 2);

SET @.y = @.y +1;

END

SET @.i = @.i +1

END;

PRINT 'Transaction Report';

SELECT t.Id, t.otherData, t.TimeRecorded, d.Description, o.TValue

FROM dbo.DeviceTranOutput o

JOIN dbo.Devices d

ON d.Id = o.IdDevice

JOIN dbo.DeviceTran t

ON t.Id = o.IdTran;

DROP TABLE dbo.DeviceTranOutput, dbo.DeviceTran, dbo.Devices;

resulting in something likeTransaction Report
Id otherData TimeRecorded Description TValue
-- - -- --
1 Data 1 2006-07-17 18:23:01.717 PDA 0
1 Data 1 2006-07-17 18:23:01.717 PBAX 1
1 Data 1 2006-07-17 18:23:01.717 PC 0
1 Data 1 2006-07-17 18:23:01.717 xxx 1
2 Data 2 2006-07-17 18:23:01.717 PDA 1
2 Data 2 2006-07-17 18:23:01.717 PBAX 0
2 Data 2 2006-07-17 18:23:01.717 PC 1
2 Data 2 2006-07-17 18:23:01.717 xxx 0
3 Data 3 2006-07-17 18:23:01.717 PDA 0
3 Data 3 2006-07-17 18:23:01.717 PBAX 1
-- result abrdiged..

storage will be better used, you have some penality in both reporting and inserting, as JOIN operations are involved, but you get far better design you can expand/modify with no worries about modifications in the device pattern...

regards

|||Thanks for that.

I'll probably adopt your suggestion.

In order to save space, I am considering only storing the ocasions when the digital values change (rather than every time step).
Using your method, I don't need to store every channel every time so that is another reason in favour of it.

It does lead to another question though..

Taking your example: say a DeviceTran record is generate every timestep for the benefit of some analogue channel 'X' that is continuously changing.
Take a digital channel called 'Y' that is only recorded every time it changes.
Then, if I do an outer join to gather all the data at every time step, I might get something like this,,,

Id TimeRecorded X Y
-- -- --
1 2006-07-17 18:23:01.000 1.0 false
2 2006-07-17 18:23:02.000 1.1 null
3 2006-07-17 18:23:03.000 1.2 null
4 2006-07-17 18:23:04.000 1.1 true
5 2006-07-17 18:23:05.000 1.4 null
6 2006-07-17 18:23:06.000 1.5 null
7 2006-07-17 18:23:07.000 1.2 false
8 2006-07-17 18:23:08.000 0.9 null
9 2006-07-17 18:23:09.000 0.8 nullFor a particular value of DATETIME I would like to get the most recent
record for Y.
A straight outer join would just show null values for those not present.
Is there a clever way of joining but using the 'last' value based on
datetime or id.

Regards
|||

hi,

I'm not sure I fully understand your requirements..

I think you mean you just avoid to insert repeted values as following..

SET NOCOUNT ON;

USE tempdb;

CREATE TABLE dbo.Devices (

Id int NOT NULL PRIMARY KEY,

Description varchar(10) NOT NULL

);

CREATE TABLE dbo.DeviceTran (

Id int NOT NULL IDENTITY PRIMARY KEY ,

otherData varchar(10) NULL,

TimeRecorded datetime NOT NULL DEFAULT GETDATE()

);

CREATE TABLE dbo.DeviceTranOutput (

Id int NOT NULL IDENTITY PRIMARY KEY ,

IdTran int NOT NULL

CONSTRAINT fk_DeviceTran_DeviceTranOutput

FOREIGN KEY

REFERENCES dbo.DeviceTran (Id),

IdDevice int NOT NULL

CONSTRAINT fk_Devices_DeviceTranOutput

FOREIGN KEY

REFERENCES dbo.Devices (Id),

TValue bit NULL

);

DECLARE @.t datetime, @.id int, @.dev int;

SELECT @.dev = 1, @.t = GETDATE();

INSERT INTO dbo.Devices VALUES ( @.dev , 'PBAX' );

INSERT INTO dbo.DeviceTran VALUES ( 'Data Pbax' , @.t);

SELECT @.id = SCOPE_IDENTITY();

INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.dev , 0);

WAITFOR DELAY '00:00:01'

SELECT @.t = GETDATE();

INSERT INTO dbo.DeviceTran VALUES ( 'Data Pbax' , @.t);

SELECT @.id = SCOPE_IDENTITY();

--INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.dev , 0);

WAITFOR DELAY '00:00:01'

SELECT @.t = GETDATE();

INSERT INTO dbo.DeviceTran VALUES ( 'Data Pbax' , @.t);

SELECT @.id = SCOPE_IDENTITY();

--INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.dev , 0);

WAITFOR DELAY '00:00:01'

SELECT @.t = GETDATE();

INSERT INTO dbo.DeviceTran VALUES ( 'Data Pbax' , @.t);

SELECT @.id = SCOPE_IDENTITY();

INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.dev , 1);

WAITFOR DELAY '00:00:01'

SELECT @.t = GETDATE();

INSERT INTO dbo.DeviceTran VALUES ( 'Data Pbax' , @.t);

SELECT @.id = SCOPE_IDENTITY();

--INSERT INTO dbo.DeviceTranOutput VALUES ( @.id , @.dev , 1);

PRINT 'Transaction Report';

PRINT '';

PRINT 'IdDevice is NULL, you can''t directly reference';

PRINT 'the devices, you have to resort on ''specific'' queries';

SELECT t.Id, t.otherData, t.TimeRecorded, d.Description,

(SELECT TOP 1 o2.TValue

FROM dbo.DeviceTranOutput o2

WHERE o2.IdDevice = d.Id

AND o2.IdTran <= t.Id

ORDER BY o2.Id DESC) AS TValue

FROM dbo.DeviceTranOutput o

JOIN dbo.Devices d ON d.Id = o.IdDevice

RIGHT JOIN dbo.DeviceTran t ON t.Id = o.IdTran

ORDER BY t.TimeRecorded;

PRINT 'as passing a [@.DeviceId] as a parameter';

DECLARE @.DeviceId int;

DECLARE @.DeviceDescription varchar(10);

SELECT @.DeviceId = 1;

SELECT @.DeviceDescription = Description

FROM dbo.Devices

WHERE Id = @.DeviceId;

SELECT t.Id, t.otherData, t.TimeRecorded,

@.DeviceDescription AS [Description],

(SELECT TOP 1 o2.TValue

FROM dbo.DeviceTranOutput o2

WHERE o2.IdDevice = @.DeviceId

AND o2.IdTran <= t.Id

ORDER BY o2.Id DESC) AS TValue

FROM dbo.DeviceTranOutput o

JOIN dbo.Devices d ON d.Id = o.IdDevice

RIGHT JOIN dbo.DeviceTran t ON t.Id = o.IdTran;

DROP TABLE dbo.DeviceTranOutput, dbo.DeviceTran, dbo.Devices;

--<-

Transaction Report
IdDevice is NULL, you can't directly reference

the devices, you have to resort on 'specific' queries

Id otherData TimeRecorded Description TValue

-- - -- --

1 Data Pbax 2006-07-18 22:56:51.810 PBAX 0

2 Data Pbax 2006-07-18 22:56:52.827 NULL NULL

3 Data Pbax 2006-07-18 22:56:53.827 NULL NULL

4 Data Pbax 2006-07-18 22:56:54.827 PBAX 1

5 Data Pbax 2006-07-18 22:56:55.827 NULL NULL

as passing a [@.DeviceId] as a parameter

Id otherData TimeRecorded Description TValue

-- - -- --

1 Data Pbax 2006-07-18 22:56:51.810 PBAX 0

2 Data Pbax 2006-07-18 22:56:52.827 PBAX 0

3 Data Pbax 2006-07-18 22:56:53.827 PBAX 0

4 Data Pbax 2006-07-18 22:56:54.827 PBAX 1

5 Data Pbax 2006-07-18 22:56:55.827 PBAX 1

but my idea is you'll go into troubles both at insert time, as you have to check if the current value is the same as the last one, and later at query time, as you miss some references..
you can scan for an older value in dbo.DeviceTranOutput of a previous transaction, but you miss the IdDevice value... if you query for a specified device then it's allright, as you pass the IdDevice as a parameter, which becames a constant, but a general purpose query to list all transactions (orderd by TimeRecorded and IdDevice) becames heavy, for every row ...
at insert time this is a heavy load as well as instead of just inserting you have to check, and this can cost a lot in real time apps..
considering you're collecting data in quiet real time, I'd go for the quicker (is it english?) way to pump data in, without trigger to filter out repeated values...

more, transactionally, it breaks a rule of atomicity of a row, as it depends on the values of previous rows... it makes all the design trickier, and of corse coding as well.. my $0.02..

regards

|||Thanks for that.
You have understood my requirements exactly.

I was hoping to save disk space by not storing all the values for a timestep that haven't changed. But you are right about breaking the rule of atomicity, because the value of a particular column now depends on its 'last' non-null value. I hadn't thought of it like that.

Whatever the gains I make in disk space I have to seriously consider the penalty in SQL complexity. Trying to extract the 'current' value for just one or maybe a handful of channels can be done (as you have shown above) but a generic query to return values for all channels might be very complex/slow.

I'll go away and think some more on it.

Thanks once again.

Regards

Monday, March 12, 2012

Optimal location for SQL USR and PWD

We have created an application in VB6 where mulitple clients access a SQL
Server 2000 database. The application uses the 3tier model. We currently hav
e
20 users, but this will probably increase exponentialy. All users uses the
same SQL username and password and we handle the security in the application
.
I need advice on where and how to store the sql login username and password.
Thank you.If you are using three tiers, the safest place for the USR and PWD is in the
middle tier.
Assuming a COM+ object, you get the administrator to assign the account and
password to the middle tier. Assuming that no one has direct access to that
machine, then the password is pretty safe.
If you manage it all from the client tier, then you must store the passwords
somewhere, encrypt it somehow, and protect the knowledge of how to decrypt
it. There are several routines available out there (try GOOGLE) that you
can use for the encrypt / decrypt.
Russell Fields
"Hugo" <Hugo@.discussions.microsoft.com> wrote in message
news:7137E0FF-D18F-4FBB-B167-EB63C93FEC44@.microsoft.com...
> We have created an application in VB6 where mulitple clients access a SQL
> Server 2000 database. The application uses the 3tier model. We currently
have
> 20 users, but this will probably increase exponentialy. All users uses the
> same SQL username and password and we handle the security in the
application.
> I need advice on where and how to store the sql login username and
password.
> Thank you.
>|||We have not moved to COM+ yet. I wanted to know if there is any standard
practice for client apps.
I will now the registry with ecryption.
Thank you.

Friday, March 9, 2012

Opinion about design needed (splitting string data)

Hi to everyone,

My problem is, that I'm not so quite sure, which way should I go.

The user is inputing by second part application a long string (let's
say 128 characters), which are separated by semiclon.
Example:

A20;BU;AC40;MA50;E;E;IC;GREEN

Now: each from this position, is already defined in any other table, as
a separate record. These are the keys lets say. It means, a have some
properities for A20, BU, aso.

Because this long inputed string, is a property of device (whih also
has a lot of different properities) I could do two different ways of
storing data:

1. By writing, in SP, just encapsulate each of the position separated
by semicolon, and write into a different table with index of device,
and the position in long stirng nearly in this way:

Major device data table
ID AnyData1 AnyData2 ... AnyData3
123 MZD12 XX77 ... any comment text
124 MZD13 XY55 ... any other comment

String data Table
fk_deviceId position value
123 1 A20
123 2 BU
123 3 AC40
....
123 8 GREEN

The device table, contains also a pointer (position), which might
change, to "hglight" specified position.

Then, I can very easly find all necessary data. The problem is, I need
to move the device record data (from other table) very often into other
history table (by each update). That will mean, that I also need to
move all these records from 1 -8 for example to a separate history
table, holding the index for a history device dataset. This is a little
inconvinience in this, and in my opinion, it will use to much storage
data, and by programming, I need always to shift this properities into
history table, whith indexes to a history table of other properities.

2. Table will be build nearly in this way:

Major device data table
ID AnyData1 AnyData2 ... AnyData3 stringProperty pointer
123 MZD12 XX77 ... any comment text A20;BU;AC40;MA50;E;E;IC;GREEN 3
124 MZD13 XY55 ... any other comment A20;BU;AC40;MA50;E;E;IC;GREEN 2

By writng into device table, there will be just a additional field for
this string, and I will have a function, which according to specified
pointer, will get me the string part on the fly, while I need it.
This will not require the other table, and will reduce the amout of
data, not a lot ... but always.
This solution, has a inconvinance, that it will be not so fast doing a
search over the part of this strings, while there will be no real index
on this.
If I woould like to search all devices, by which the curent pointer
value is equal GREEN, then I need to use function for getting the
value, and this one will be not indexed, means, by a lot amount of
data, might be slow.

I would like to know Your opinion about booth solutions.
Also, if you might point me the other problems with any of this
solution, I might not have noticed.

With Best Regards

MatikMatik (marzec@.sauron.xo.pl) writes:

Quote:

Originally Posted by

1. By writing, in SP, just encapsulate each of the position separated
by semicolon, and write into a different table with index of device,
and the position in long stirng nearly in this way:
>
Major device data table
ID AnyData1 AnyData2 ... AnyData3
123 MZD12 XX77 ... any comment text
124 MZD13 XY55 ... any other comment
>
String data Table
fk_deviceId position value
123 1 A20
123 2 BU
123 3 AC40
...
123 8 GREEN
>
The device table, contains also a pointer (position), which might
change, to "hglight" specified position.

This is the normal design in this situation.

Quote:

Originally Posted by

This design violates a basic principle in relational design: no repeating
groups.

Every rule is made to break, and I have occasionally put repeating groups in
the database I maintain, but this is a clearcut case: don't even think
about it. This sort of data is very difficult to work with in a
relational database, simply because it's not meant that you should
store data in this way.

Quote:

Originally Posted by

With a sub-table you need to repeat the ID. There will also be a cost
of two bytes for the length of each column. There is also the cost for
the field number, but since you don't have any semi-colon, this is a
net cost of one byte. There is also some overhead for each row. But
all and all, I would say that the overhead is about neglible.

Quote:

Originally Posted by

and by programming, I need always to shift this properities into
history table, whith indexes to a history table of other properities.

Don't really know what you mean here.

For completeness sake I should say that there is a third alternative,
and that is one table, but eight columns. This could also be considered
a repeating group. Then again, if the different fields represents
different attributes, it isn't really an repetition. This solution
is better my opinion than a seprated list, but the pointer you talk
about may be more difficult to implement.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||Erland Sommarskog wrote:

Quote:

Originally Posted by

Matik (marzec@.sauron.xo.pl) writes:

Quote:

Originally Posted by

Quote:

Originally Posted by

>and by programming, I need always to shift this properities into
>history table, whith indexes to a history table of other properities.

>
Don't really know what you mean here.

Probably something along the lines of: (oversimplified for brevity)

insert into FooHistory select * from CurrentFoo
delete from CurrentFoo|||Then maybe you could create a view for the second table.

Matik wrote:

Quote:

Originally Posted by

Hi to everyone,
>
My problem is, that I'm not so quite sure, which way should I go.
>
The user is inputing by second part application a long string (let's
say 128 characters), which are separated by semiclon.
Example:
>
A20;BU;AC40;MA50;E;E;IC;GREEN
>
Now: each from this position, is already defined in any other table, as
a separate record. These are the keys lets say. It means, a have some
properities for A20, BU, aso.
>
Because this long inputed string, is a property of device (whih also
has a lot of different properities) I could do two different ways of
storing data:
>
1. By writing, in SP, just encapsulate each of the position separated
by semicolon, and write into a different table with index of device,
and the position in long stirng nearly in this way:
>
Major device data table
ID AnyData1 AnyData2 ... AnyData3
123 MZD12 XX77 ... any comment text
124 MZD13 XY55 ... any other comment
>
String data Table
fk_deviceId position value
123 1 A20
123 2 BU
123 3 AC40
...
123 8 GREEN
>
The device table, contains also a pointer (position), which might
change, to "hglight" specified position.
>
Then, I can very easly find all necessary data. The problem is, I need
to move the device record data (from other table) very often into other
history table (by each update). That will mean, that I also need to
move all these records from 1 -8 for example to a separate history
table, holding the index for a history device dataset. This is a little
inconvinience in this, and in my opinion, it will use to much storage
data, and by programming, I need always to shift this properities into
history table, whith indexes to a history table of other properities.
>
2. Table will be build nearly in this way:
>
Major device data table
ID AnyData1 AnyData2 ... AnyData3 stringProperty pointer
123 MZD12 XX77 ... any comment text A20;BU;AC40;MA50;E;E;IC;GREEN 3
124 MZD13 XY55 ... any other comment A20;BU;AC40;MA50;E;E;IC;GREEN 2
>
By writng into device table, there will be just a additional field for
this string, and I will have a function, which according to specified
pointer, will get me the string part on the fly, while I need it.
This will not require the other table, and will reduce the amout of
data, not a lot ... but always.
This solution, has a inconvinance, that it will be not so fast doing a
search over the part of this strings, while there will be no real index
on this.
If I woould like to search all devices, by which the curent pointer
value is equal GREEN, then I need to use function for getting the
value, and this one will be not indexed, means, by a lot amount of
data, might be slow.
>
I would like to know Your opinion about booth solutions.
Also, if you might point me the other problems with any of this
solution, I might not have noticed.
>
With Best Regards
>
Matik

|||First of all, thank you for your reply!

Now, some additional explenations maybe:

That was just an example, with 8 positions separated by semicolon as a
one property. The problem is, there number of this is various. That's
why, I couldyn't solve issue with fix number of column.

With shifting data into history, I've ment, that by each change of data
in primary table, whole record should be copied to the history table
(nearly same construction as primary table).
This is than an issue with the second table, storing semicolon
separated field in one column (splitted) in different table. This need
to be shifted then also, to a second historical table.

Of course, I could ommit using 'working' table, and have only history,
with inserts, and having a primary table containing a pointer to last -
newest record as my primary table, to get the newest record.
The problem is, I'm afraid a little of performance, sice there is all
other actions done on the primary table (select, searches aso.)
Having a big historical table, I will still need to get countinous
joins, to get the newest record, and even having a good indexing and
relation set up, it might be slow while table can be big.

This semicolon devided string, as example was shown pretty simmilar,
but it can be also various:

A10;B13;c20;bubu;lala;GREEN;RED
A13;BUBU;GREEN;YELLOW;mama
C25;YELLOW
BLUE;pleple;B13
aso.

The pointer I was talking about, is just a index, to which position in
this semicolon devided string, is curently activated.

Best regards

Matik|||Matik wrote:

Quote:

Originally Posted by

The problem is, I'm afraid a little of performance,

This has "premature optimization" written all over it. Build the
database cleanly first; then, if you /actually/ have performance
issues, then consider how to improve it (but breaking 1NF with "a;b;c"
type columns should still be a last resort).|||Matik (marzec@.sauron.xo.pl) writes:

Quote:

Originally Posted by

I'm not sure that I see the problem. With a regular design, you would
have two tables for current data, and two tables for historical data.

Quote:

Originally Posted by

Like Ed said, get the design right first, and do performance tuning
when everything else is working. But some basic ideas for performance
are good when designing for performance. For instance no repeating
groups (i.e. semicolon-separated lists.)

Quote:

Originally Posted by

>
Of course, I could ommit using 'working' table, and have only history,
with inserts, and having a primary table containing a pointer to last -
newest record as my primary table, to get the newest record.
The problem is, I'm afraid a little of performance, sice there is all
other actions done on the primary table (select, searches aso.)
Having a big historical table, I will still need to get countinous
joins, to get the newest record, and even having a good indexing and
relation set up, it might be slow while table can be big.
>

The way to optimise is with good indexes and good query design. You say
"it might be slow" so obviously you haven't reached that stage yet. On
the other hand you know for sure that a redundant copy of the data will
have an additional performance cost, both for updates and queries.

--
David Portas, SQL Server MVP

Whenever possible please post enough code to reproduce your problem.
Including CREATE TABLE and INSERT statements usually helps.
State what version of SQL Server you are using and specify the content
of any error messages.

SQL Server Books Online:
http://msdn2.microsoft.com/library/...US,SQL.90).aspx
--

Friday, March 30, 2012

Optimizing table in mssql

Optimizing Queries generated at runtime

Wednesday, March 28, 2012

Optimizing Database

Optimizing cursor performance?

Optimizing cursor performance?

Optimizing cursor performance?

Monday, March 26, 2012

optimized name-address-zipcode-region design

Optimize Temp tables

Optimize Temp tables

Optimize Temp tables

Friday, March 23, 2012

optimizations job for db maintenance plan failed

optimizations job for db maintenance plan failed

Wednesday, March 21, 2012

Optimization issue

Tuesday, March 20, 2012

Optimization

optimistic locking

Monday, March 19, 2012

Optimising Select statements which has a LIKE where clause.

optimising a table with lots of boolean fields

Monday, March 12, 2012

Optimal location for SQL USR and PWD

Friday, March 9, 2012

Opinion about design needed (splitting string data)

OpenXML Identity Insert

Blog Archive

About Me