Week 4 (6/5 - 6/11)

Schedule

1. Mon, June 5
   • Notes:
     • whether serialize skeleton data
2. Tues, June 6
   • Meeting: 3:00 - 3:30 pm
   • Notes:
     • serialization
     • byte[] to create pictures
     • pointcloud data from depth frame, array of points
     • client in java, decode
3. Wed, June 7
   • Meeting: 4:00 - 4:30 pm (skype)
   • Notes:
     • byte array: temporally hardcode the size
     • depth frame 16-bit unsigned integers long instead of ushort
     • for the color frame: image formate type table & check method
4. Thurs, Jun. 1
   • Thur email
   • Notes:
     • didn't get callback on KinectColorFrameReceived
       • This method is assigning an event handle to the event kinect.ColorFrameReady (in KinectController.cs)
       • suggestion: FrameSourceTypes + read bools from preference file specified for each stream type
       • fixed by ditching all the code that slows down color frame processing (using frame - dispose)
     • did get body data, which is passed after convert from string to byte[]; not with color frame data, which is sent directly as byte[]
5. Fri, Jun. 2
   • Meeting: 1:00 - 1:15 pm
   • Notes:
     • trime down / downsample color image
     • http://kinectservice.codeplex.com/SourceControl/latest
     • reducing the color frame resolution to a lower resolution so as to enable faster processing

DataContractAttribute Class is in the System.Runtime.Serialization namespace.

But you should add reference to System.Runtime.Serialization.dll

And assembly isn't referenced by default, so you have to go to References -> Add Reference in Solution Explorer and add an assembly reference

[{"LeanTrackingState":2,"Lean":{"X":-0.0046482766,"Y":0.307799637},"IsRestricted":false," IsTracked":true," TrackingId":72057594037928293,"ClippedEdges":8," Engaged":0," HandRightConfidence":1," HandRightState":1," HandLeftConfidence":1," HandLeftState":1," Appearance":{"WearingGlasses":0},"Activities":{"EyeLeftClosed":0,"EyeRightClosed":0,"MouthOpen":0,"MouthMoved":0,"LookingAway":0},"Expressions":{"Neutral":0,"Happy":0},"JointOrientations":{"SpineBase":{"JointType":0,"Orientation":{"X":-0.05233748,"Y":0.9205873,"Z":-0.0939059556,"W":0.375448316}},"SpineMid":{"JointType":1,"Orientation":{"X":-0.053282842,"Y":0.9167501,"Z":-0.0933728,"W":0.384722918}},"Neck":{"JointType":2,"Orientation":{"X":-0.0698294,"Y":0.907467246,"Z":-0.133008137,"W":0.39234665}},"Head":{"JointType":3,"Orientation":{"X":0.0,"Y":0.0,"Z":0.0,"W":0.0}},"ShoulderLeft":{"JointType":4,"Orientation":{"X":0.642500937,"Y":-0.5883303,"Z":0.382910371,"W":-0.307310164}},"ElbowLeft":{"JointType":5,"Orientation":{"X":0.93015945,"Y":0.0258215778,"Z":0.34632596,"W":-0}}}}]

PixelFormats

PixelFormats.Bgra32

Gets the Bgra32 pixel format. Bgra32 is a sRGB format with 32 bits per pixel (BPP). Each channel (blue, green, red, and alpha) is allocated 8 bits per pixel (BPP).

pick different colors when visualizing depth data

The depth data is 16 bits, so we use a USHORT to read it in.

We usually represent the furthest distances using black color and the nearest distances using white color.

Kinect: a 1920×1080 (AKA 1080p) 30 fps 16:9 camera

Create a static class and add the following method:

public static class Extensions // must be static !

{

    /// <summary>

    /// Convert a kinect color frame to a bitmap

    /// </summary>

    /// <param name="frame">the frame to convert</param>

    /// <returns>the bitmap</returns>

    public static ImageSource ToBitmap(this ColorFrame frame)

    {

        int width = frame.FrameDescription.Width;

        int height = frame.FrameDescription.Height;

        PixelFormat format = PixelFormats.Bgr32;

        byte[] pixels = new byte[width * height * ((format.BitsPerPixel + 7) / 8)];

        if (frame.RawColorImageFormat == ColorImageFormat.Bgra)

        {

            frame.CopyRawFrameDataToArray(pixels);

        }

        else

        {

            frame.CopyConvertedFrameDataToArray(pixels, ColorImageFormat.Bgra);

        }

        int stride = width * format.BitsPerPixel / 8;

        return BitmapSource.Create(width, height, 96, 96, format, null, pixels, stride);

    }

}