|
Ah, thanks.
I thought it might look through and see that o1 doesn't have any references to it, so it would make that eligible. But it would see that o2 has a ref from o1 which hadn't been collected yet, etc.
Seems like the GC is smarter then that .
|
|
|
|
|
We have a need to send very large files to an internal FTP server. These files are videos that may last as long as an hour. All of the FTP sample I find have the standard code in them:
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Timeout = CONNECTION_TIMEOUT;
request.ReadWriteTimeout = CONNECTION_TIMEOUT;
request.UsePassive = false;
if (credentials != null)
request.Credentials = credentials;
StreamReader sourceStream = new StreamReader(source);
byte[] fileContents = Encoding.UTF8.GetBytes(sourceStream.ReadToEnd());
sourceStream.Close();
request.ContentLength = fileContents.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(fileContents, 0, fileContents.Length);
requestStream.Close();
This logic gives us an out-of-memory condition with very large files. What I'd like to do is have it send the file as a multi-part file sent as small chunks instead of a single large stream.
Does anyone know how this is implemented?
Software Zen: delete this;
|
|
|
|
|
I see no reason to go for complex schemes, however it makes no sense to me that you first read the entire file into memory, and then send it in a single write. Both operations are stream operations, so use them as such, with small amounts, and in a loop.
And then, I'm puzzled by the Encoding.UTF8.GetBytes statement; there is no text involved anywhere, so why would one need an Encoding? All it takes is byte transfers: byte array in, byte array out.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
Well, Luc, you took my post too literally.
The code I posted was one of the first examples I quickly grabbed.
Essentially all the code is as follows:
get a stream reader, read the data into a byte array, create a stream writer, feed it the byte array.
Our issue as I noted in the example is that the byte[] buffer = stream.ReadToEnd() always throws an out of memory exception.
I was ready that for very large files you can do a multi-part so that you read and transmit small packets and the FTP server puts the packets together into the original massive file. That is what I'm looking for so that we do not blow out our memory again.
Software Zen: delete this;
|
|
|
|
|
you should read my post and take it literally.
Do NOT use ReadToEnd(); use a loop, and read and write smaller chunks.
That is called streaming.
Luc Pattyn [My Articles] Nil Volentibus Arduum
Fed up by FireFox memory leaks I switched to Opera and now CP doesn't perform its paste magic, so links will not be offered. Sorry.
|
|
|
|
|
You just mean chunking the file? I've not heard of 'multi-part FTP'. If that is what you mean, it's a very common pattern; roughly speaking:
Stream inStream = GetInputStream();
Stream outStream = request.GetRequestStream();
const int blocksize = 8192;
byte[] buf = new byte[blocksize];
int read = 0;
while(0 < (read = inStream.ReadBytes(buf, blocksize)))
outStream.WriteBytes(buf, read);
|
|
|
|
|
So every new website launched (revamped or whatever) these days ought to have good Bug Tracking/Reporting systems in place as no website is perfect first time. However, I have not recently come across a system that actually works well and would apply to my websites situation. I am trying to make a cloud app where users access stuff entirely online through my site but obviously as I have written the (primitive) access program to them myself, it is going to have bugs. I therefore need a system to allow bug reporting/viewing/tracking or whatever you want to call it.
So what do people think is the best way of doing it?
Obviously the basic requirements are:
- A list of existing bug reports (including which are being worked on)
- Some method of submitting a bug report
- A way for me (the developer) to respond effectively to bug reports and tell users that I am doing so.
Could anyone offer any insite/comment/help as to how best to do this?
Thanks very much,
Ed
Edit: This thread has been reposted by OP in ASP.Net forum - please see here for full thread:
New Thread[^]
modified 6-Feb-12 11:17am.
|
|
|
|
|
I think this question belongs in the ASP.NET Forum[^].
Unrequited desire is character building. OriginalGriff
I'm sitting here giving you a standing ovation - Len Goodman
|
|
|
|
|
Yes my apologies ( ) I had written it as a more general question for C# but then realised that was a rubbish question (too general) so rewrote it but forgot to change forum... There doesn't appear to be a way for me to shift it to the other forum though?! Not without deleting and re-posting... 
|
|
|
|
|
Edward Nutting wrote: Not without deleting and re-posting
Unfortunately a feature that does not yet exist. However, you could add a comment to that effect and re-post in ASP.NET forum.
Unrequited desire is character building. OriginalGriff
I'm sitting here giving you a standing ovation - Len Goodman
|
|
|
|
|
Good idea! I shall do that
Thanks,
Ed
|
|
|
|
|
I'd suggest Spolsky's FogBugz[^]. You might find additional suggestions on our Free Tools[^] forum.
Bastard Programmer from Hell
|
|
|
|
|
Thanks but this doesn't really help (unless I have totally missed something from FogBuzz). I need either a system that can integrate as part of my website so that anyone can log bugs (without needing to log in) or just some ideas as to what makes a good bug tracker.
|
|
|
|
|
Aw, you're right - FogBugz is a bit more than an issue-tracker. It's the most complete bug-tracker that I've worked with so far, and customers can enter issues using the website or by email. There's a two-minute overview on SO[^].
Bastard Programmer from Hell
|
|
|
|
|
Aaah thanks! The main site really didn't make it clear to me that users can do stuff all online! And by email would be fantastic! I will try and integrate it immediately Thank you very much! Any tips on installation/integration?
Thanks very much,
Ed
|
|
|
|
|
Edward Nutting wrote: Any tips on installation/integration?
They've got a special section on plugins[^], hook up SourceSafe[^]..
Aw, with integration you mean tips on embedding it in the website, of course, not integration with other apps. Ehr, no tips there, I'm a WinForms person. My idea of embedding a website is an IFRAME
FogBugz isn't free, and there may be good and free alternatives that fit the bill that I don't know of. Give it a day or two, as there may be more ideas from other forum-members.
Bastard Programmer from Hell
|
|
|
|
|
which one is faster?
fetching processed data from database
or processing data at asp page
for example i have two dates - start date and end date. on almost all the pages i need total number of days between these two dates. also all dates between them etc. so should i store total number of days in DB or process it on each page where i need them?
________________
Waiting for answer...
We can have facts without thinking but we cannot have thinking without facts.
****************
|
|
|
|
|
Calculating the number of dates between two dates, or determining each date between two dates, is such a trivial task that you will almost certainly get worse performance retrieving them from a database. Put another way; A database connection and command execution will be many times slower than counting from startDate to endDate.
For refernce, the two method look like this:
public static double NumberOfDaysInRange(DateTime startDate, DateTime endDate)
{
return endDate.Subtract(startDate).TotalDays;
}
public static IEnumerable<DateTime> DatesInRange(DateTime startDate, DateTime endDate)
{
for(var date = startDate; date<endDate; date = date.AddDays(1))
yield return date;
}
Live example: http://rextester.com/JKDTC31937[^]
|
|
|
|
|
thanks. easy methods to calculate numofdays. ...
________________
We can have facts without thinking but we cannot have thinking without facts.
****************
|
|
|
|
|
Use the TimeSpan class to calculate date differences.
DateTime dt1 = ...;
DateTime dt2 = ...;
TimeSpan ts = dt1 - dt2;
int days = ts.Days;
|
|
|
|
|
|
Since you are probably going to have to retrieve the start or end date from a database anyway, there is going to be little or no real performance difference between them. However, it is always worth storing dates as dates and doing any necessary match later. If nothing else, it allows you to change how you do the math without affecting the data, or to filter based on end date. For example, today it could be "number of days between" you need - tomorrow it could be "number of complete weeks between". If you store only the days difference, you cannot calculate the weeks accurately for existing data.
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
|
|
|
|
|
thanks, liked your answer. much understandable and close to my question.
i have two dates "StartDate" & "EndDate" stored in database as datetime datatype. everywhere i need to show these dates i need to show total number of days as well in between. (there is no such comparison as of now).
right now we are only storing dates not total number of days. so what the programmer doing is. . . he is calculating total number of days on each page on the basis of start date and end date fetch from database.
i think in this case, it is good to store number_of_days in DB rather to calculate them everywhere.
Right?
________________
We can have facts without thinking but we cannot have thinking without facts.
****************
|
|
|
|
|
Probably not - it is storing redundant information (i.e., you can calculate it from the two dates) so it introduces the possibility of the two sets of info being out of step - which can cause some horrible-to-spot intermittent bugs. If you are going to be using it really, really often (like on 50,000 records per second) then it could be worth the extra storage and space, but otherwise the really time consumer is going to be setting up the DB connection and accessing the data anyway. If you are really worried, you could move the calculation to the SQL Server by doing it as part of the query but frankly, given this is a website thing, if the extra cost of a DateTime subtraction is a major problem to your webserver then you seriously need to look at an upgrade! (Or a massive overhaul of your website as a whole)
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
|
|
|
|
|
 "hi there, thanks a lot for reply to this question again.
i read ur answer so many times then it was cleared. your way of communication is quit difficult to understand for someone who is not English.
well can you please reply me again of below question.
i am really confused how to design the database for the below requirement. please read the requirement summary below and answer these question.
This is a wizard kind of data forms:
- at first page, I have a date range, that i am storing in database as datetime datatype "StartDate" & "EndDate"
- On the 2nd step. I am showing total_number_of_days
- On 3rd Step, I am showing both dates with total_number_of_days. Here I need a Grid filled with all the dates (days) between "StartDate" & "EndDate" with day name and day number.
- On 4th step I again have the dates with total_num_of_days, but here i also have date range selection. this section should be beteen the "StartDate" & "EndDate" stored previously.
- Now after selecting date range from 4th step, i need to show a tabular data with all the dates selected in 4th step.
I need to store some other related information on each step for each day.
Should I store all dates in a separate table? so that all related information can be connected properly by a foreign key relation.
otherwise If i will not store each day separately how can i make a relation between the day and the other related data?
eagerly waiting for your reply...
"
It's difficult to answer with a absolute "do it this way", but in general:
If you are storing the same information for different days, then you need a row for each day (and hence a new table), referencing back to your date range row. I would think from the info you have given that you need a table to hold the start and end date (but not to hold the number of days between - that can be calculated each time) and a table to hold each day's information, with a forieng key back to the start and end row ID.
If the information for a day consists of the same information types repeated, then that should have it's own table, referencing back to the Day table by a foreign key.
Try it on sheets of paper! You will see what I mean.
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
|
|
|
|