Click here to Skip to main content
15,904,351 members
Home / Discussions / Database
   

Database

 
PinnedHOW TO ANSWER A QUESTION PinPopular
Chris Maunder16-Jul-09 3:10
cofounderChris Maunder16-Jul-09 3:10 
PinnedHow to get an answer to your question PinPopular
Chris Maunder10-Nov-05 16:30
cofounderChris Maunder10-Nov-05 16:30 
QuestionVb6 and sqlserver 2000 Pin
ahmed ali Apr20217-May-24 17:31
ahmed ali Apr20217-May-24 17:31 
AnswerRe: Vb6 and sqlserver 2000 Pin
Victor Nijegorodov7-May-24 20:07
Victor Nijegorodov7-May-24 20:07 
AnswerRe: Vb6 and sqlserver 2000 Pin
Richard Deeming7-May-24 21:32
mveRichard Deeming7-May-24 21:32 
GeneralRe: Vb6 and sqlserver 2000 Pin
jschell9-May-24 15:35
jschell9-May-24 15:35 
GeneralRe: Vb6 and sqlserver 2000 Pin
Richard Deeming9-May-24 20:58
mveRichard Deeming9-May-24 20:58 
GeneralRe: Vb6 and sqlserver 2000 Pin
jschell11-May-24 9:43
jschell11-May-24 9:43 
GeneralRe: Vb6 and sqlserver 2000 Pin
Victor Nijegorodov11-May-24 21:10
Victor Nijegorodov11-May-24 21:10 
GeneralRe: Vb6 and sqlserver 2000 Pin
Dave Kreskowiak10-May-24 13:31
mveDave Kreskowiak10-May-24 13:31 
GeneralRe: Vb6 and sqlserver 2000 Pin
Richard Deeming12-May-24 21:22
mveRichard Deeming12-May-24 21:22 
GeneralRe: Vb6 and sqlserver 2000 Pin
Dave Kreskowiak13-May-24 3:03
mveDave Kreskowiak13-May-24 3:03 
QuestionSQLite and Firefox Browser 68.0.2 (note: current FFB is now 72.0.2) and what I stand to lose if I update. Pin
RedDk6-May-24 8:56
RedDk6-May-24 8:56 
AnswerRe: SQLite and Firefox Browser 68.0.2 (note: current FFB is now 72.0.2) and what I stand to lose if I update. Pin
Peter_in_27806-May-24 15:18
professionalPeter_in_27806-May-24 15:18 
AnswerRe: SQLite and Firefox Browser 68.0.2 (note: current FFB is now 72.0.2) and what I stand to lose if I update. Pin
Richard Deeming6-May-24 21:44
mveRichard Deeming6-May-24 21:44 
QuestionLooking for a Better Database Solution for Trading Systems Pin
Rene Rose 202120-Apr-24 5:53
Rene Rose 202120-Apr-24 5:53 
AnswerRe: Looking for a Better Database Solution for Trading Systems Pin
Jeremy Falcon20-Apr-24 8:15
professionalJeremy Falcon20-Apr-24 8:15 
Are there no other NoSQL databases you can try besides LiteDB? If you go the relational route and attempt to normalize your flattened data being inserted, it's not going to get faster. Not sure if your important is mean to be realtime or not. So, this is just general-purpose ideas...

These would be your options:

1) Are you using connnection pooling or opening an new connection with every insert? Perhaps that's the bottleneck. LiteDB may not have the concept of connections at all, but if it does that's the first place to check.

2) Can you toss in more threads to this import process? Will LiteDB even handle concurrency or will it choke?

3) Determine why your current LiteDB is choking. Is the bottleneck in your code or the DB? Is there a locked transaction not working? Is it thread safe? Are you using more than one thread getting locked? etc.

4) If the above doesn't work, find a different NoSQL DB that doesn't choke. Get a real one and not one shipped as a DLL once meant for concurrency as it'll have the best throughput even on a single user. I've used MongoDB, I'm sure there are others.

5) If none of that works, then go download MariaDB (MySQL fork), but make sure this import table is using ISAM storage. SQL Server will not be anywhere as fast as this as it doesn't allow you to choose storage engines. This will be for unnormalized data only that supports nothing fancy like triggers, constraints, and foreign keys, but ISAM is fast preciously for that reason. And, you can always have another import process/ETL transform the data if needed in non-realtime.

6) Since it's just an import, the fastest (but last resort) solution would be to just write out the data into an appended binary file. You'd still need a process to import it into an actual DB, but that can be offloaded so the original process isn't bottlenecked.

Rene Rose 2021 wrote:
One possible solution would be to distribute this data class across multiple tables to avoid duplicate entries. However, I'm concerned that this may cause performance issues when inserting several hundred thousand data points.
Duplication isn't an issue as long as your tables aren't indexed... for writing. Most NoSQL databases don't index, so chances are you're good. Reading is a different story however.
Jeremy Falcon

AnswerRe: Looking for a Better Database Solution for Trading Systems Pin
RedDk20-Apr-24 8:33
RedDk20-Apr-24 8:33 
AnswerRe: Looking for a Better Database Solution for Trading Systems Pin
jschell23-Apr-24 14:25
jschell23-Apr-24 14:25 
QuestionRun an external program C# from within the Oracle Developer Pin
Member 162354202-Apr-24 10:37
Member 162354202-Apr-24 10:37 
AnswerRe: Run an external program C# from within the Oracle Developer Pin
jschell2-Apr-24 14:18
jschell2-Apr-24 14:18 
QuestionDisplay records based on selected date range in months within same year? Pin
samflex21-Jan-24 9:28
samflex21-Jan-24 9:28 
AnswerRe: Display records based on selected date range in months within same year? Pin
jschell22-Jan-24 5:50
jschell22-Jan-24 5:50 
GeneralRe: Display records based on selected date range in months within same year? Pin
samflex22-Jan-24 8:35
samflex22-Jan-24 8:35 
GeneralRe: Display records based on selected date range in months within same year? Pin
Victor Nijegorodov22-Jan-24 10:10
Victor Nijegorodov22-Jan-24 10:10 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.