Visual Studio 2011 Beta
I started development on my Gwn.Kinect.Desktop library with the Depth stream. I was surprised to find that approximately 20% of the time when I started my application the Kinect would generate a HRESULT: 0x80070016 error, red light would display for a few seconds, and then the sensor would shut down – I had to disconnect and reconnect to get it working again. This happened with no consistency other than it would consistently happen, i.e., sometimes it work work 6/7 times in a row and others it would fail after the first or second launch of the application.
It is important to note that this is being developed independent of a UI, I am developing for maintainability, extensibility and reusability. As the framework nears completion I will plug in the UI components so I have no idea if the initialization techniques that I was using (from the book identified below) will even work. I should note that the fix was to apply initialization techniques used in the SDK examples. 2012.14.12 Update I have the ColorStream plugged in (WebCam) to verify components are working – change set 97421.
2012.14.12 UPDATE – Although this blog demonstrates how to avoid slow initialization times I have been unsuccessful eliminating the HRESULT issue running in Debug mode. I have to emphasize “Debug mode” because I had been running the Kinect Explorer (demo) from runtime executable with no issues (so I was assuming I was doing something wrong). Turns out that after loading the Kinect Explorer source, and running in debug mode (F5) I could not run the Kinect Explorer more than 5 consecutive times without an HRESULT error. I did find that I could consistently “start without debugging (Ctrl+F5)” without the error so I tried this with my SolrContribDesktop application and so far have not been able to get it to crash (I tried 10 consecutive times – this is a record). So far my workaround will be to run without debugging (Ctrl-F5) and then attach the debugger. Crashes when starting without debug (Ctrl+F5) also…
Microsoft Bug Reported HERE <== UPDATED 5/14/2012, Microsoft reported that this is a Windows bug.
Figure 1 – HRESULT error information
In running the sample applications that accompanied the Windows Kinect sensor I noted two things – the sensor initialized considerably faster and it consistently loaded 2012.04.22.. It was time to debug this issue….
My http://SolrContrib.CodePlex.com project (change set 97373) has a SolrContribDesktop application using the (decoupled) Gwn.Kinect.Desktop module. This is a Multi Targeted application, however the Kinect module will only be available to desktop apps (since Kinect hardware is not supported by Metro).
The KinectPresenter of this module (using the MVP-VM pattern) is currently configured to implement the Depth and Skeleton features of the Kinect (not ColorStream). Below I show the code for the command that will handle initializing and starting of the Kinect sensor under this configuration:
Figure 2 – Command that configures and starts sensor
Since my code followed the patterns indicated by the book **Beginning Kinect Programming with the Microsoft Kinect SDK (pg 10 for ColorStream and pg 50 for DepthStream) with no success, in regards to the initialization steps, I decided to add the missing ingredients that seemingly allowed the Kinect Explorer sample to work (the Kinect Explorer accompanies the SDK). I enabled the color stream and used constructor parameters to see what would happen. I was pleasantly surprised to find that not only did my crashing occur less frequently, now the sensor was initializing with the same performance that I was seeing in the Kinect Explorer demo!!! To appreciate the difference you should watch the video clip by clicking on figure 4 and THEN watch the video clip shown in figure 5, you will see the performance I’ve been dealing with since I started development of the Kinect – it is significant difference!!!
** I strongly recommend purchasing this book if you are going to do any serious Kinect programming – I found it invaluable in the learning/developing process. Visual Studio 2011 Beta was released since this book was written and many things have changed (been improved). Note: I found that their downloaded samples have the constructor parameters.
Note: The highlighted changes in the upper panel in figure 3 are the ONLY changes I made to get the performance and reliability benefits, they are executed “before” the KinectSensor.Start() is called on line 308.
Figure 3 – Sensor.cs source code excerpt
Note: The sensor starts initializing when the IR Projector (red light) turns on. You will find that during the initialization process it will turn on, turn off and then turn back on – it is then ready for use. After enabling the color stream the delay between the times the projector turns on is significantly shorter. See the difference between video clips by clicking figure 4 and figure 5.
Figure 4 - After enabling ColorStream and using constructor parameters
After watching the following video in figure 5, watch the video in figure 4 again to appreciate how much faster it is.
Figure 5 - Without the ColorStream enabled (no constructor parameters)
Figure 6 - Kinect will shut down and report “Not Powered” after crashing (20% of the time)