Click here to Skip to main content
15,559,819 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
honey the codewitch29-Jan-21 10:42
mvahoney the codewitch29-Jan-21 10:42 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
Sander Rossel29-Jan-21 12:19
professionalSander Rossel29-Jan-21 12:19 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
honey the codewitch29-Jan-21 12:30
mvahoney the codewitch29-Jan-21 12:30 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
k505429-Jan-21 11:15
mvek505429-Jan-21 11:15 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
Sander Rossel29-Jan-21 12:19
professionalSander Rossel29-Jan-21 12:19 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
k505430-Jan-21 4:50
mvek505430-Jan-21 4:50 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
trønderen29-Jan-21 13:30
trønderen29-Jan-21 13:30 
GeneralRe: strpbrk() in Microsoft's VCLib is *slow* Pin
honey the codewitch29-Jan-21 14:38
mvahoney the codewitch29-Jan-21 14:38 
I am using real world data collected from an online repository at TMDB.com. I have 200kB of actual data from their repository, and then I synthesized 20MB of similar data in the same schema. I could have downloaded 20MB of JSON from TMDB.com. The only problem is then I'm downloading 20MB of data from tmdb.com and their rate limiting will hate me.

Now, for a real world scenario, where you're actually using TMDB's data, you'll likely end up mirroring their repository as you retrieve parts of it. For example, their repository contains every show and movie you'll find at IMDb.com, but in JSON format. Now if I only want shows from 2019, i can get those, but the point is this process is like fetch on request, and then cache. If you were to retrieve all the data then the entire repository would be mirrored locally.

It is from this mirror that i'd want to extract data.

So yes, that's a real world scenario. I even have a C# library that does this for tmdb.com already, but not using this json parser, which is in C++.

I've profiled it using the GNU profiler on linux, but nothing else.

Most of the function time is in strpbrk() at least for long scans.

More importantly, I know my actual throughput. I'm currently getting 2/3 of the throughput I got on a linux machine, on a windows machine whose hardware is maybe 10 times as fast or more.

And i know what function primarily impacts that throughput because I've already profiled. Smile | :)

It's skipToAny() which in the best case, uses strpbrk() - it can't on arduinos but it will on windows.
Real programmers use butterflies

RantMicrosoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 2:17
mvahoney the codewitch29-Jan-21 2:17 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
OriginalGriff29-Jan-21 2:20
mveOriginalGriff29-Jan-21 2:20 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 2:35
mvahoney the codewitch29-Jan-21 2:35 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Randor 29-Jan-21 2:50
professional Randor 29-Jan-21 2:50 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 3:18
mvahoney the codewitch29-Jan-21 3:18 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Randor 29-Jan-21 3:27
professional Randor 29-Jan-21 3:27 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 3:38
mvahoney the codewitch29-Jan-21 3:38 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Randor 29-Jan-21 3:51
professional Randor 29-Jan-21 3:51 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 4:01
mvahoney the codewitch29-Jan-21 4:01 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Randor 29-Jan-21 4:25
professional Randor 29-Jan-21 4:25 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
honey the codewitch29-Jan-21 4:33
mvahoney the codewitch29-Jan-21 4:33 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Stuart Dootson29-Jan-21 6:52
professionalStuart Dootson29-Jan-21 6:52 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
dandy7229-Jan-21 4:23
dandy7229-Jan-21 4:23 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
Randor 29-Jan-21 4:34
professional Randor 29-Jan-21 4:34 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
dandy7229-Jan-21 6:18
dandy7229-Jan-21 6:18 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
charlieg3-Feb-21 17:23
charlieg3-Feb-21 17:23 
GeneralRe: Microsoft's Clownish Developer Prompt Pin
charlieg3-Feb-21 17:17
charlieg3-Feb-21 17:17 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.