Comparing to the competition
I was checking the most popular download sites to see if they updated our Watchdog - O - Matic program when I saw I had a new competitor. They've been selling a shareware program that tries to do the same thing as Watchdog - O - Matic since January. So I was wondering how their program performed. It's only natural that competitors check each other’s program and compare theirs with it. What's not so frequent is that someone posts test results and their thoughts about it online. Yes, I know this might help my competitors fix their problems. I'm such a helpful guy. I will try to give you an honest opinion, but I might be biased sometimes.
It's easy to see that their program is made with mine as an example. Several features in the application are named the same way. And even parts of the manual and website are copied and changed a little.
E.g. I had an option before 5.00 (removed because it's obsolete in the new version):
"Don't start if already running but watch the active process"
Their option is:
"Don't start another copy if already running"
They do have a nice looking manual though. This is a part where I still need some improvements. Personally, I think it's not necessary to spend my time writing huge manuals IF you create a GUI that's easy to use. This is why I have been adding tooltips so the user doesn't have to refer to the manual all the time.
As far as I can tell, the new program doesn't actually 'debug' the target, but is watching the CPU and Memory activity. This is similar to other competitors, and I'm expecting the same problems with regards to detecting crashes. I configured the program to check my own crash application, and these are the results.
Crash in the main thread: a crash dialog appears requiring a user action, and a 'hanging application' is detected. The crash application is freezed by windows so that's why the program detects a not responding application. The program tries to close our application, but is unable to do that due to the dialog (even with the terminate option enabled). So our crash test program just sits there. It crashed, and it stays dead while the program is waiting to restart the crashed application.
They acknowledge the problem in their forum, and suggest that the user should disable the crash dialogs (although this doesn't work with the steps they suggest). Users shouldn't change settings like these if it can be avoided. Certainly if it is a setting outside of your own application (e.g. Windows).
A crash in a different thread is not detected. Apparently the program only checks if the main thread is running. Unfortunately, this isn't enough. You need to dig a lot deeper if you want to detect problems in multithreaded applications. Otherwise, you won't even notice that there is a problem and the target application isn't running like it should.
The 'not responding check' is working. The program checks the CPU usage and the message pump of the main thread, which is sufficient. This is exactly how we check for hanging applications.
CPU usage: when an application is using 100% on a dual core, then the program measures a 50% usage so no problem is detected. I haven't tried the detection on a single core CPU, but I suspect that this will work. I do wonder what will happen if your application is using 100% for a limited time, which might be normal behaviour. It's probably an 'all or nothing' option that can't be tuned to detect normal from abnormal CPU usage.
Memory usage works, although the program seems to check the virtual memory size instead of real memory usage.
While using the program, I discovered some strange things in the user interface. E.g. viewing your application statistics displays the following screen, which is probably a debug window?
It's not all bad. They do have an interesting feature that's been on my todo list for a while. This will be an incentive to implement new features and clean up that todo list of mine.
It's easy to see that their program is made with mine as an example. Several features in the application are named the same way. And even parts of the manual and website are copied and changed a little.
E.g. I had an option before 5.00 (removed because it's obsolete in the new version):
"Don't start if already running but watch the active process"
Their option is:
"Don't start another copy if already running"
They do have a nice looking manual though. This is a part where I still need some improvements. Personally, I think it's not necessary to spend my time writing huge manuals IF you create a GUI that's easy to use. This is why I have been adding tooltips so the user doesn't have to refer to the manual all the time.
As far as I can tell, the new program doesn't actually 'debug' the target, but is watching the CPU and Memory activity. This is similar to other competitors, and I'm expecting the same problems with regards to detecting crashes. I configured the program to check my own crash application, and these are the results.
Crash in the main thread: a crash dialog appears requiring a user action, and a 'hanging application' is detected. The crash application is freezed by windows so that's why the program detects a not responding application. The program tries to close our application, but is unable to do that due to the dialog (even with the terminate option enabled). So our crash test program just sits there. It crashed, and it stays dead while the program is waiting to restart the crashed application.
They acknowledge the problem in their forum, and suggest that the user should disable the crash dialogs (although this doesn't work with the steps they suggest). Users shouldn't change settings like these if it can be avoided. Certainly if it is a setting outside of your own application (e.g. Windows).
A crash in a different thread is not detected. Apparently the program only checks if the main thread is running. Unfortunately, this isn't enough. You need to dig a lot deeper if you want to detect problems in multithreaded applications. Otherwise, you won't even notice that there is a problem and the target application isn't running like it should.
The 'not responding check' is working. The program checks the CPU usage and the message pump of the main thread, which is sufficient. This is exactly how we check for hanging applications.
CPU usage: when an application is using 100% on a dual core, then the program measures a 50% usage so no problem is detected. I haven't tried the detection on a single core CPU, but I suspect that this will work. I do wonder what will happen if your application is using 100% for a limited time, which might be normal behaviour. It's probably an 'all or nothing' option that can't be tuned to detect normal from abnormal CPU usage.
Memory usage works, although the program seems to check the virtual memory size instead of real memory usage.
While using the program, I discovered some strange things in the user interface. E.g. viewing your application statistics displays the following screen, which is probably a debug window?
It's not all bad. They do have an interesting feature that's been on my todo list for a while. This will be an incentive to implement new features and clean up that todo list of mine.
Labels: Watchdog
posted by Wouter Dhondt at
19:00
0 Comments:
Post a Comment
<< Home