|
Hi Richard - many thanks for the quick reply...
Richard Deeming wrote: Don't measure the performance of your code using DateTime ; use the Stopwatch class[^] instead.
I'm not measuring the performance per-se, just getting an idea, and in any case that part of the code won't be in the prod version...
Richard Deeming wrote: Did you make sure to "warm up" the code before you timed it, and run the test over many iterations?
Absolutely - same perf over many MANY runs
Richard Deeming wrote: Since you know the maximum number of items you'll be adding to the list, you should set its capacity when you create it:
List<object> entityBatch = new List<object>(batchSize);
If you don't set the capacity, it starts at 4 and doubles each time it runs out of space.
Valid point - made the change and no real effect - still around 12s
Richard Deeming wrote: You need to check the value returned from enumerator.MoveNext ; if it returns false , you've reached the end of the sequence. Your code currently continues adding the final items from the sequence to the list until you reach the batch size.
Again, valid point, but in this instance it's not the problem - I can be certain that it's not at the end.
Richard Deeming wrote: You should consider using proper generic types, rather than a List<object> ; that way, you avoid having to cast the items back to the correct type when you read the batches.
Also a valid point, but not possible in this instance - it's in an abstract base class, which operates over several types of similar but not related types and exists in a (necessarily) singleton class....my thinking is leading me to the point that it may well be an area to look at, but given that IEnumerator.Current returns an object type anyway I'm not sure.
Richard Deeming wrote: It's probably best to wrap this sort of thing up as a extension method
Ultimately that's a possibility, but until I can bottom out the cause of the several orders of magnitude difference it's sort of neither here or there - appreciate the comment however.
Many thanks again - I'll have to keep digging into it and see what's what!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
RichardGrimmer wrote: not possible in this instance - it's in an abstract base class, which operates over several types of similar but not related types and exists in a (necessarily) singleton class....
So you can't do something like this?
public abstract class MyBaseClass
{
public void ProcessList<T>(IEnumerable<T> listToProcess) where T : MyBaseClass
{
...
}
}
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Unfortunately not (admittedly not thought it all the way through, but the types in question don't share a common base class)....I do appreciate that I could re-structure so that they do however, but again, given the fact that the problem manifests itself a s difference of several orders of magnitude, I'm in a "bigger fish to fry" type of situation - thanks again for the suggestion however - I may well end up doing something similar "just for grins" and see what happens....just for background, the code is used to index items from several different data sources into an Elastic Search instance, hence the object type being used....
C# has already designed away most of the tedium of C++.
|
|
|
|
|
 What do your classes actually look like? I took your description and built these out:
public class ChildItem
{
public ChildItem()
{
Count = int.MaxValue;
LongCount = long.MaxValue;
Nullable2 = int.MaxValue - 3;
Nullable3 = Nullable2.Value - 2;
}
public int Count { get; set; }
public long LongCount { get; set; }
public int? Nullable1 { get; set; }
public int? Nullable2 { get; set; }
public int? Nullable3 { get; set; }
}
public class MasterList
{
private List<ChildItem> children = new List<ChildItem>();
public MasterList()
{
for (int i = 0; i < 6; i++)
{
children.Add(new ChildItem());
}
Master1 = "Hi";
Master2 = "Hi";
Master3 = "Hi";
Master4 = "Hi";
Master5 = "Hi";
Master6 = "Hi";
Master7 = "Hi";
Master8 = "Hi";
Master9 = "Hi";
Master10 = "Hi";
Master11 = "Hi";
Master12 = "Hi";
}
public IEnumerable<ChildItem> Children => children;
public string Master1 { get; set; }
public string Master2 { get; set; }
public string Master3 { get; set; }
public string Master4 { get; set; }
public string Master5 { get; set; }
public string Master6 { get; set; }
public string Master7 { get; set; }
public string Master8 { get; set; }
public string Master9 { get; set; }
public string Master10 { get; set; }
public string Master11 { get; set; }
public string Master12 { get; set; }
}
public class TestIt
{
private List<MasterList> items = new List<MasterList>(700000);
public TestIt()
{
for (int i = 0; i < 700000; i++)
{
items.Add(new MasterList());
}
Console.WriteLine("Ready");
}
public void StartTest(string text = "Dummy run")
{
var enumerator = items.GetEnumerator();
enumerator.MoveNext();
List<object> entityBatch = new List<object>();
Stopwatch sw = new Stopwatch();
sw.Start();
for (int iterStep = 0; iterStep < 700000; iterStep++)
{
entityBatch.Add(enumerator.Current);
enumerator.MoveNext();
}
sw.Stop();
Console.WriteLine(text);
Console.WriteLine("Total time {0}", sw.ElapsedMilliseconds);
}
} Running this through a few times consistently returns a time of less than 50 milliseconds.
This space for rent
|
|
|
|
|
Hi Pete,
So the classes in question represent a name and a collection of dates of birth associated with the name...classes below;
public partial class DateOfBirth
{
public int DateOfBirthId { get; set; }
public Nullable<int> Day { get; set; }
public Nullable<int> Month { get; set; }
public Nullable<int> Year { get; set; }
public long PepDeskRecordId { get; set; }
}
public class ExpandedFoo
{
public string FooId { get; set; }
public string BarId { get; set; }
public string Title { get; set; }
public string Gender { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string FullName { get; set; }
public string OtherNames { get; set; }
public string Country { get; set; }
public string BlahId{ get; set; }
public string CombinedName { get; set; }
public IEnumerable<DateOfBirth> DateOfBirths { get; set; }
public ExpandedFoo()
{
this.DateOfBirths = new List<DateOfBirth>();
}
}
(Apologies for the naming, edited for a variety of corporate reasons!)
All in all pretty basic classes, and similar to those you've put together...
C# has already designed away most of the tedium of C++.
|
|
|
|
|
OK - I've cracked the issue, but I'm not even going to pretend I know why...it appears it was down to how I was populating my lists....
The pop code originally looked like;
var expandedEntities = Foontities.Select(entity => new ExpandedFoo
{
FooId = entity.fooId.ToString(CultureInfo.InvariantCulture),
BarId = entity.BarId.ToString(CultureInfo.InvariantCulture),
Title = entity.Title,
Gender = entity.Gender,
FirstName = entity.FirstName,
LastName = entity.LastName,
FullName = entity.FullName,
OtherNames = entity.OtherNames,
CombinedName = entity.CombinedName,
Country = entity.Country,
BlahId= entity.BlahId.ToString(),
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.FooId== entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
}).ToList()
});
Which was causing the 12s time to extract 1000 records, but (as mentioned, this is to index into Elastic Search), converting to Json and building a bulk command using PlainElasic.net was <.2s....
Removing the ToList(), to make the DOB population code;
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.PooId == entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
})
Fixed the extraction problem and took the time to milliseconds - BUT caused the Json conversion and bulk command building to take (coincidentally) 12s...
HOWEVER, doing;
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.FooId == entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
}) as List<DateOfBirth>
Made it run in the region of .001s for both the extract and the convert and bulk command creation....
I think I need to look at the IL and see what's going on!
Many thanks to both you and Richard for all your help - if I can figure out the issue / differences, I'll post back and let you both know!
Thanks again gents!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Just as a test, go back to your ExoandedFoo and change the IEnumerable<DataOrBirth> into a List<DataOfBirth> and populate it using the ToList() method.
This space for rent
|
|
|
|
|
Hi Pete,
Actually I was all sorts of wrong - on deeper investigation, the as statement was failing silently (as it of course should) - a cast rather than the as did indeed throw exceptions left right and centre.
Not a problem, I've taken a different tack now...I may at a later date circle back around and look at it in more detail, but for now it'll do as it is.
Many MANY thanks for your time on this one
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Hi Richard,
I've just posted an update above - very odd, but thought you might like to take a look - it was (apparently) down to how I was populating my data...
Many thanks for all your help and suggestions!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Dear All,
I am making a C# window program with a function playing a local video using window medial Player. I need a function like capture a image frame from the playing video using a button click that will transfer capture directly into a picturebox, and then save into local desk with a button(save button).
Thank you for your help in advance,
|
|
|
|
|
And what code do you have so far? What, exactly, are you expecting us to provide? I can make an awful lot of assumptions here but assumptions are a bad thing.
This space for rent
|
|
|
|
|
Pete O'Hanlon wrote: I can make an awful lot of assumptions here And the first one is probably correct. 
|
|
|
|
|
|
So I'm still playing around with Rx, and usually, I found it very neat:
private void Window_Loaded(object sender, RoutedEventArgs e)
{
var Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
}
I do however find it more useful in most cases to send the value using the Select statement. But when I pass this around, I can't unsubscribe to it like so:
private void Window_Loaded(object sender, RoutedEventArgs e)
{
var Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
Moves.Dispose();
}
How can I dispose of the subscribtion?
|
|
|
|
|
Nevermind I'm stupid
private void Window_Loaded(object sender, RoutedEventArgs e)
{
IObservable<Point> Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
var DisposableSubscribtion = Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
DisposableSubscribtion.Dispose();
}
|
|
|
|
|
public string ConfidenceConnected(string filename)
{
string ConfidenceConnected = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory) + "\\image\\ConfidenceConnected.exe";
int seedX = 100;
int seedY = 78;
string file = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory) + "\\image\\" + filename + " output.dcm" + " " + seedX + " " + seedY;
//string Out = " output"+ filename+".dcm";
string cmd = "/c" + ConfidenceConnected + file;
System.Diagnostics.Process process = new System.Diagnostics.Process();
process.StartInfo.UseShellExecute = false;
process.StartInfo.FileName = "cmd.exe";
process.StartInfo.Arguments = cmd;
process.StartInfo.RedirectStandardOutput = true;
string return_string = " ";
try
{
process.Start();
return_string= process.StandardOutput.ReadToEnd();
}
catch (Exception e)
{
return_string = e.Message;
}
return return_string;
}
|
|
|
|
|
And what problem do you have? That's just a code dump.
This space for rent
|
|
|
|
|
Spaces are a problem here.
First off, you need a space between ConfidenceConnected and file , and before ConfidenceConnected :
string cmd = "/c" + ConfidenceConnected + file;
Otherwise your command string becomes:
/cC:\path\image\ConfidenceConnected.exeC:\path\image\filename output.dcm 100 78
Second, if you want paths with spaces, they need to be delimited with double quotes:
/c "C:\path\image\ConfidenceConnected.exe" "C:\path\image\filename output.dcm" 100 78
But please try to use Path.Combine[^] instead of building your paths manually!
It's a good idea to build your parts, then finalize with a String.Format:
string cmd = string.Format("/c \"{0}\" \"{1}\" {2} {3}", ConfidenceConnected, file, seedX, seedY);
It makes it a lot easier to see the overview of what you are trying to achieve.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
The other day I posted about a problem[^] with FileSystemWatcher raising events before the file is completely copied into the watched folder.
Thanks for Gerry Schmitz[^] I think I've come up with a workable solution:
I would appreciate any comments on this idea.
[1] The FileSystemWatcher detects the file. It's a large file that take a while to get fully copied into the folder:
private void OnCreated(object sender, FileSystemEventArgs e)
{
if (HasAnotherFileEventOccuredRecently(e.FullPath))
{
return;
}
QueueFile(e);
}
The HasAnotherFileEventOccuredRecently method is called to see if this is the first time a FileSystemWatcher event has been raised for the file. Multiple events are raised for the file, so if any events for the file have already been called, then ignore them. I didn't include this code for brevity.
[2] Next QueueFile is called:
private void QueueFile(FileSystemEventArgs fileSystemEventArgs)
{
var worker = new BackgroundWorker();
worker.DoWork += Worker_DoWork;
worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
_queue.Add(worker);
worker.RunWorkerAsync(fileSystemEventArgs);
}
The BG Worker snippets...
private void Worker_DoWork(object sender, DoWorkEventArgs e)
{
var args = (FileSystemEventArgs)e.Argument;
if (args != null)
{
while (!IsFileAvailable(args.FullPath))
{
}
}
e.Result = args;
}
and
private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
var args = (FileSystemEventArgs)e.Result;
var worker = _queue.FirstOrDefault(x => x == sender);
if (worker != null)
{
worker.Dispose();
_queue.Remove(worker);
}
this.OnCreated(args);
}
[3] and finally the IsFileAvailable method:
public bool IsFileAvailable(string filename)
{
var results = false;
try
{
using (var inputStream = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.None))
{
results = inputStream.Length > 0;
}
}
catch (Exception)
{
results = false;
}
return results;
}
If it's not broken, fix it until it is
modified 22-Feb-16 16:59pm.
|
|
|
|
|
|
Am trying to get a list of all sql sever instance in my network using this code but i get an empty list
var dt = SqlDataSourceEnumerator.Instance.GetDataSources();
to verify i put in CMD this "sqlcmd -L" and "osql -L" and i get the list.
so what am doing wrong, i dont know if OS matter am using Win10. Also someone know how to do this using dlls thar sqlserver have?
thanks.
|
|
|
|
|
The code I use is pretty much identical:
SqlDataSourceEnumerator instance = SqlDataSourceEnumerator.Instance;
DataTable dt = instance.GetDataSources();
foreach (System.Data.DataRow row in dt.Rows)
{
Console.WriteLine("ServerName = {0}", row["ServerName"]);
}
I just tried it under Win 10, and it works fine for me.
What does the debugger say?
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
i use your code now and it's the same result.
DataTable dt = instance.GetDataSources();
dt.Rows.Count = 0
any idea why?
|
|
|
|
|
How's your network? Can you see any instances in the VS Solution Explorer pane or SSMS?
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
as i said before i run this command in CMD "sqlcmd -L" and i get the list, in this case for now just found my server. In "SSMS" and "VS server exporer" i see my server too.
|
|
|
|
|