This is why I don't like jumping into someelse's argument, it becomes mine
Yes, I've followed the posts and just to be sure, I went back and re-read all of yours specifically (you should do the same to refresh your memory). You never mention going back to a stack buffer "after the code is good". In fact, quite the opposite, you were adament about buffers always being from the heap, always, always. You repeated that quite often. So the resposes where to those statements.
What you say about locking mecnahisms isnt the case though. The memory manager
deals with that, not the code, so it doesnt add complication.
Well that's a convenient hand wave, blaming the underlying function rather than the caller who invokes it. The point is that memory allocation at that level is costly and that even if the "memory manager" has to deal with the locking, etc, you still are resposible for chosing a methodology that invokes that call over stack allocation. So, the introduction of the overhead is your choice, the kernel code is just giving you what you asked for. Don't blame it.
Plus IO is the real bottle neck
IO is a "wait state" event and not chewing up cpu cycles, which was what this discussion was all about. The kernel / application is free to do other things while IO is going on using any number of asynchronous IO techniques. If you wish to now have a discussion on all the things that affect application / kernel performance, we can do that too.
Stability is more imporant than speed. Always.
Only an idiot would argue in favor of "instability". Of course stability is important. In fact, if you're getting paid to do code, your client / employer will assume stability and will find someone else to deliver it if you fail. So most don't even bother listing stability as a priority, it's assumed you will deliver it. On the other hand, many will list speed as the priority, depending on the application. Imagine trying to defend a radar application that is too slow to catch all the incoming phase radar data by saying "but it's stable!!".
Vectors use the heap effectively because they allocate memory ahead of time instead of doing a new every time. They effectively get around the slowness at the expense of having some extra memory allocated ahead of time. Think he refuses to hear other people's arguments at all.
Oh, and by the way, if you regularly need a set size of heap buffer you can preallocate and manage yourself. In the kernel you can use look aside lists to do the same. Very good they are too. I work a lot in the kernel. Believe me, stability is more important.
Because it is very easy to get sh*t data in code. With sh*t data it is very easy to overun a buffer. A stack cprruption is very hard to debug. A verifier barf on a heap overrun is very easy to debug.
It is that simple. Make your life simple.
Make the code 100%. Dont worry about speed. Modern languages have already killed performabnce to such an extent and memory is so cheap that for a commercial app on a PC it is wrong to think of speed and resources,
On a dedicated computer on a satellite with expensive limited HW it is a different thing, But think of this. You develope your code using heap based buffers, and when it is 100% solid, you then use stack based buffers.
See where I am going? It is called clever engineering. Develope the product. Make it perfect. Use any technique you can to achieve that, and the verifier tools from microsoft are very very good.
I think some of the others have addressed your question directly already (as to the error), but another interesting follow-up for you comes up. Why do you need buffers that large? Maybe you need to think of a different way of achieving your end-solution.
You said your app is dialog based, this also means that you have multiple threads, even though you may not be creating them manually. You can in fact allocate arrays of arbitrary size on the stack, but this is true only for the main thread. If you try to allocate on the stack of a thread, then this stack will be limited by the amount defined upon creation of that thread! If you do not create that thread yourself, or do not pass a stack size, then a default value will be assumed, and this value is probably smaller than what you wish to allocate.
The solution is to either
1. Allocate the array on the main thread and pass a pointer to it to the other thread, or
2. Allocate on the heap, rather than on the stack, or
3. Find out how to manually override the stack size of the thread in question (if that is even possible)
Having said that, the few answers I did bother to skim over seem to be mostly mistaken in their analysis and reasoning, even though some of the suggestions (such as allocating on the heap) might work.
Hope this helps.
P.S.: While I stated the stack of the main thread can be arbitrarily big, you still may have to tell the compiler this, via options, as Chuck already pointed out. I just tested on VS 2003 and VS 2010, and both define a default stacksize of 1 MB. I've found that, on a 12GB system I could increase the stack to 10GB and define an array of 1000 million doubles on the stack (in the main thread). (in VS, you can find that option under Linker/System/Stack Reserve Size)
I have a CWinApp based DLL that worked fine in VS6. Since converting it to VS2010, I can't get it to access the InitInstance function (or even the constructor for that matter).
I can see using Process Explorer that it loads at the correct time, but it doesn't initialise properly. I can access its functions using GetProcAddress, so I can probably work around it, but I'd like to know what the problem is!
(It is difficult to debug because I don't load it directly - I tell a third party DLL about it and (presumably) that DLL uses LoadLibrary and GetProcAddress to load my DLL).
What version of Studio was the old DLL built with? Under certain conditions, old DLLs that were built with different studio versions will fail to load (simple DLLs that use plain C/C++ usually don't have this issue but more complex DLLs do).
I don't recall exactly the issue that leads to this problem, may be the fact that you're using different MFC versions and there was some sort of non-reverse compatible change along the way or something similar, but the easiest solution if you have the source, is to upgrade the source to compile in the same studio version as your main executable.