Data Streams

1. Enable any or all of the data streams (color, depth, and skeleton) and then start the sensor to begin streaming data out.

Frame Source Types

Identify which streams are needed and Open a stream

MultiSourceFrameReader myReader;

// open reader for frame source, specify which streams to be used

myReader = mySensor.OpenMultiSourceFrameReader(FrameSourceTypes.Body | FrameSourceTypes.Depth | FrameSourceTypes.Color);

2. Get the next frame of data

Register an event that fires when data is ready / Set up a call back

myReader.MultiSourceFrameArrived += MultiSouceFrameArrived;

Associate event handler by using "+="

Implement an event handler

private void MultiSouceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e)

...

sender object is the Kinect Sensor that fired the event

var frame = e.FrameReference.AcquireFrame();

using (BodyFrame bdFrame = frame.BodyFrameReference.AcquireFrame())

...

AcquireFrame gets the actual color frame from the reference
using statement: frame will be automatically disposed of when done using it

Preallocate buffers to hold sensor data

byte[] colorFramePixels = new byte[fd.Width * fd.Height * 4];

store the pixel data and then allocate the memory array
4 bytes will store the data from one pixel

Get the new data for each stream each frame

frame.CopyConvertedFrameDataToArray(colorFramePixels, ColorImageFormat.Bgra);

Copies the color frame data converted into the requested format to a byte array.

frame.CopyRawFrameDataToArray(colorFramePixels);

Copies the color frame data to a byte array.

color image frame is in BGRA format
BGRA has 4 bytes of data per pixel

3. Store the data in the bitmap

Create a bitmap to store the data

https://msdn.microsoft.com/en-us/library/hh973069.aspx

FrameDescription

FrameDescription fd = sensor.InfraredFrameSource.FrameDescription;

infraredFrameData - Intermediate storage for receiving frame data from the sensor
infraredPixels - Intermediate storage for frame data converted to color pixels for display
bitmap – The WritableBitmap which will replace the contents of an image in xaml

irBitmap = new WriteableBitmap(fd.Width, fd.Height);

WriteableBitmap

api: https://msdn.microsoft.com/en-us/library/system.windows.media.imaging.writeablebitmap.aspx
get much better performance creating a WriteableBitmap only when the pixel format changes, such as when you start the application.
BitmapSource.Create method is used to do that, the parameters of the methods are;
- pixelWidth: The width of the bitmap.
- pixelHeight: The height of the bitmap.
- dpiX: The horizontal dots per inch (dpi) of the bitmap.
- dpiY: The vertical dots per inch (dpi) of the bitmap.
- pixelFormat: The pixel format of the bitmap. (BGR, CMYK, Black etc)
- Palette: The palette of the bitmap.
- Pixels: An array of bytes that represents the content of a bitmap image.
- Stride: The stride of the bitmap. (Number of bytes allocated for one scan line ofthe bitmap).
For greater control over updates, and for multi-threaded access to the back buffer, use the following workflow.
1. Call the Lock method to reserve the back buffer for updates.
2. Obtain a pointer to the back buffer by accessing the BackBuffer property.
3. Write changes to the back buffer. Other threads may write changes to the back buffer when the WriteableBitmap is locked.
4. Call the AddDirtyRect method to indicate areas that have changed.
5. Call the Unlock method to release the back buffer and allow presentation to the screen.

reference: https://msdn.microsoft.com/en-us/library/hh973069.aspx#CS_BitmapStore

Create a bitmap to store the data

Create a bitmap to store the pixel data.

  private WriteableBitmap colorBitmap;

  this.colorBitmap = new WriteableBitmap(this.sensor.ColorStream.FrameWidth, this.sensor.ColorStream.FrameHeight, 96.0, 96.0, PixelFormats.Bgr32, null);

A WriteableBitmap is a Windows Presentation Foundation (WPF) construct, available in .NET Framework 4, that allows access to the bitmap bits. You could create a Bitmap every frame, but you will get much better performance creating a WriteableBitmap only when the pixel format changes, such as when you start the application.

Store the data in the bitmap

After getting the color data and saving it in the WriteableBitmap, draw the WriteableBitmap.

      if (colorFrame != null)

        // Write the pixel data into our bitmap

        this.colorBitmap.WritePixels(

          new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),

          this.colorPixels,

          this.colorBitmap.PixelWidth * sizeof(int),

0);

Use the WriteableBitmap.WritePixels method to copy the data from the local array to the WriteableBitmap.

Report abuse