Click here to Skip to main content
15,845,469 members
Home / Discussions / C#
   

C#

 
QuestionWhat is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop2-Jun-21 7:58
Alex Dunlop2-Jun-21 7:58 
AnswerRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 8:08
mveDave Kreskowiak2-Jun-21 8:08 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop2-Jun-21 8:13
Alex Dunlop2-Jun-21 8:13 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Victor Nijegorodov2-Jun-21 8:48
Victor Nijegorodov2-Jun-21 8:48 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 8:50
mveDave Kreskowiak2-Jun-21 8:50 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Gerry Schmitz2-Jun-21 9:07
mveGerry Schmitz2-Jun-21 9:07 
AnswerRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop2-Jun-21 8:12
Alex Dunlop2-Jun-21 8:12 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 8:22
mveDave Kreskowiak2-Jun-21 8:22 
And this is why I said do it in a stored procedure in the database.

For every record you want to insert, why on earth would you read all of the existing data? Yes, this get's slower and slower every time you insert a record. All that data transfer will take more and more time.

It sounds like you're not even caching the data you read and insert! If you're going to do something like this, you would normally just read the database once, keep all that data, then add new records to the cache and have the cache update the database.

This has a downside though. You really cannot use it in a multi-user environment, and there is a limit to the amount of data you can keep in memory, so it doesn't scale well at all.

Doing it in the database removes reading all that data over and over again, and removes the memory size limits.

DO IT IN THE DATABASE WITH A STORED PROCEDURE!

GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop2-Jun-21 8:34
Alex Dunlop2-Jun-21 8:34 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 8:43
mveDave Kreskowiak2-Jun-21 8:43 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop2-Jun-21 8:48
Alex Dunlop2-Jun-21 8:48 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 8:51
mveDave Kreskowiak2-Jun-21 8:51 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Gerry Schmitz2-Jun-21 9:00
mveGerry Schmitz2-Jun-21 9:00 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Dave Kreskowiak2-Jun-21 9:02
mveDave Kreskowiak2-Jun-21 9:02 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Gerry Schmitz2-Jun-21 9:09
mveGerry Schmitz2-Jun-21 9:09 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop3-Jun-21 3:06
Alex Dunlop3-Jun-21 3:06 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Alex Dunlop3-Jun-21 3:05
Alex Dunlop3-Jun-21 3:05 
GeneralRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Richard MacCutchan3-Jun-21 4:12
mveRichard MacCutchan3-Jun-21 4:12 
AnswerRe: What is the best way for saving huge amount of data into SQL CE file while keeping data uniqueness? Pin
Richard Deeming2-Jun-21 22:41
mveRichard Deeming2-Jun-21 22:41 
QuestionHow to increase filtering speed in DataGridView? Pin
Alex Dunlop29-May-21 23:37
Alex Dunlop29-May-21 23:37 
SuggestionRe: How to increase filtering speed in DataGridView? Pin
Richard MacCutchan29-May-21 23:45
mveRichard MacCutchan29-May-21 23:45 
AnswerRe: How to increase filtering speed in DataGridView? Pin
OriginalGriff30-May-21 3:07
mveOriginalGriff30-May-21 3:07 
GeneralRe: How to increase filtering speed in DataGridView? Pin
Alex Dunlop31-May-21 1:58
Alex Dunlop31-May-21 1:58 
GeneralRe: How to increase filtering speed in DataGridView? Pin
OriginalGriff31-May-21 3:14
mveOriginalGriff31-May-21 3:14 
GeneralRe: How to increase filtering speed in DataGridView? Pin
Alex Dunlop31-May-21 4:59
Alex Dunlop31-May-21 4:59 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.