CSCI 520 Assignment 2: Motion Capture Interpolation

This is an old assignment. Do not use.

Due Wed Mar 9, 2022, by 11:59pm



Overview

In this assignment, you will implement three interpolation schemes to interpolate human motion data obtained from an optical mocap system. The human model (skeleton) is represented using a hierarchy, composed of a root node and several children nodes, corresponding to the various joints. The optical mocap system generates two data files: ASF file, which encodes the skeleton kinematics (lengths of bones, degrees of freedom, etc.) and AMC file, which stores the motion (translation, joint angles) in time (at 120 frames per second). The AMC file contains the translation of the root node (xyz coordinates), rotation of the root node (in Euler angles), and the joint angles (for the various joints, in Euler angles). We provide starter code that can parse ASF and AMC files and store the data into a datastructure. The starter code also includes an OpenGL motion capture player that renders the skeleton (parsed from the ASF file), and plays back the motion from an AMC file. You will be given several ASF files and AMC files corresponding to different motion capture sequences (walking, dancing, running, martial arts). For each motion sequence, you will then form a subsequence, by dropping certain frames (described below). You will use this subsequence to create an interpolated sequence of the original length, and analyze and plot the difference to the original motion.


Source: Wikipedia

Requirements

The AMC files use Euler angles to represent joint rotations. The starter code can load these Euler angles into a datastructure in memory. Your program should take the motion from the input AMC file, and then form the keyframes for interpolation. The keyframes are obtained from the original motion by dropping N consecutive frames, where N is a given input parameter. For example, suppose the input motion has 30 frames, labeled 0,1,...,29. Suppose N=6. Then, the keyframes are frames 0,7,14,21,28. Your program must use interpolation to compute new frames 1,2,...,6, 8,9,...,13, 15,16,...,20, 22,23,...,27. Set frame 29 to be identical to the input frame 29. Thus, your program produces a motion sequence of the same length as the original. Depending on the value N, and the choice of the particular interpolation technique, the interpolated motions will more or less closely resemble the input motion. For example, it is expected that quaternion interpolation produces motions that more closely resemble the input than Euler interpolation. You will analyze the different interpolation techniques in your report (more below).

The starter code can perform linear interpolation in Euler space (see interpolator.cpp), which also serves as an example for how to access motion capture joint data and create/set new motion sequences. Your task is to do the following:


Submission

Please submit the following:

Please submit the assignment using the USC Blackboard. In order to do so, create a single zip file containing your entire submission. Submit this file to the blackboard using the "Attach File" field. After submission, please verify that your zip file has been successfully uploaded. You may submit as many times as you like. If you submit the assignment multiple times, we will grade your LAST submission only. Your submission time is the time of your LAST submission; if this time is after the deadline, late policy will apply to it.

Note: The instructors will read your code. Quality of your code and comments might affect your grade. This is an individual assignment. Your code might be compared to code of other students using a code comparison tool.


Starter code and data

You can download the starter code and data here.

Code setup instructions

====================================================================================
How to build fltk in Windows, Linux, MAC OS X
====================================================================================
On Windows and Linux, please follow the corresponding readme 
files in the folder: fltk-1.3.2
Note: On Linux, you may need to install the "autoconf" tool.

On Max OS X, you should use fltk-1.3.5. Therefore, follow
the instructions in the fltk-1.3.5 folder. Also, you need
to modify two Makefiles:
%homeworkFolder%/mocapPlayer-starter/Makefile
%homeworkFolder%/mocapPlayer-starter/Makefile.FLTK
Open each file in a text editor, and change fltk-1.3.2 to fltk-1.3.5 .

Note: There are many projects in FLTK library. This homework
is only dependent on fltk and fltkgl projects. Students can choose 
to only compile those two projects in the Visual Studio project 
of the FLTK library.

Note: If you want to compile the homework under the release 
configuration, you should first compile the FLTK projects in release. 
Same goes for the debug configuration.

Note: Both the main homework project and FLTK projects work only 
on the Win32 platform. Please do not use other platforms (e.g. X64).

====================================================================================
How to build starter code in Linux, MAC OS X (Assuming fltk has been compiled)
====================================================================================
1) Enter the %homeworkFolder%/mocapPlayer-starter
2) make

====================================================================================
How to build starter code using Visual Studio 2017 (Assuming fltk has been compiled)
====================================================================================
1) Open the project file in homework folder: IDE-starter/VS2017/mocapPlayer.sln
2) Choose Debug/Release mode
3) Compile project: mocapPlayer
4) Compile project: interpolate

Because there are many versions of Windows 10 (different build versions), 
you may get an error when compiling: ... selecting "Retarget solution". 
This can be solved, as the message says, by right-clicking the solution 
and selecting "Retarget solution".

Source code structure and description

The code consists of several classes. We only provide a brief description here. For more detailed description of each class, see the corresponding .h files. When the "Load Skeleton" button is pressed, the selected ASF file is parsed, and skeleton loaded is into memory and saved into an instance of the skeleton class. When the "Load Motion" button is pressed, the AMC file is read and saved into an instance of the motion class.


Description of motion capture ASF and AMC file formats

The starter code reads and displays the data from the optical motion capture system (ASF and AMC). The world coordinate system is Y up.

Skeleton file

The skeleton file (ASF) describes how the bones in the skeleton are connected, and their degrees of freedom (see Figure 1 for a representative skeleton). All bones are described in the ":bonedata" section of this file. Each bone has the following fields:

begin
id bone_id                  //Unique id for each bone
name bone_name        //Unique name for each bone
direction dX dY dZ    //Vector describing direction of the bone in world coordinate system
length 7.01722           //Length of the bone
axis 0 0 20 XYZ         //Rotation of local coordinate system for
                                   //this bone relative to the world coordinate
                                   //system. In AMC file the rotation angles
                                    //for this bone for each time frame will be
                                   //defined relative to this local coordinate
                                    //system (see Figure 2)
dof rx ry rz                //Degrees of freedom for this bone.
limits (-160.0 20.0)
            (-70.0 70.0)
            (-60.0 70.0)
end

The parent/child relationship among all bones in skeleton hierarchy is defined in the ":hierarchy " section of the ASF file. Each line list the parent first, then its children.
For example:
"root lhipjoint rhipjoint lowerback"
Root is a parent and lhipjoint, rhipjoint, lowerback are root's children.

The following picture shows the skeleton hierarchy as defined in a representative ASF file (length of the bones is not up to scale).

Figure 1. The skeleton hierarchy, in particular name and id for each bone as defined in a representative ASF file.


Figure 2. The names and local coordinate system for some of the bones.

For a longer introduction on the mocap file format, you can look at http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/ASF-AMC.html.

Motion file

The motion file (AMC) gives each bone's rotation angles (in Euler angles) relative to the bone's reference local coordinate system. The bone's reference local coordinate system is defined in the skeleton (ASF) file. One entry is given for each frame, at a frame rate of 120 fps.

The ":FORCE-ALL-JOINTS-BE-3DOF" keyword in the header (if present), is not a standard AMC keyword. We added this keyword because the result of quaternion interpolation may produce three-dimensional Euler angles, even when original Euler angles were only one or two-dimensional. You do not need to worry about this keyword. The "interpolate" driver (starter code) will create it automatically as needed, and the "mocapPlayer" will correctly load any motion files that use this keyword.


Motion capture player instructions

The "mocapPlayer" player can play back any motion capture sequence (AMC file), on any skeleton (ASF file). A simple camera control is provided. You can drag the left mouse button to move the camera, middle mouse button to zoom in/out, and the right mouse button to rotate the camera.

Below is the picture of GUI for the program (by Yili Zhao and Jernej Barbic; first version by Steve Lin, Alla Safonova, Kiran Bhat).

Displaying the animation

You first need to load the skeleton (ASF) file. Then load the motion (AMC) file. You can now view the motion using the player buttons. You can save the current frame as an image by pressing the Screenshot button. To record a sequence of images (to make a video), first press record (R) button, then play the animation. All the frames played while the record button is enabled will be saved to disk as images.

Displaying two or more motions simultaneously

You can also display two motions on top of each other, say, to visualize the difference between the input motion and the interpolated motion. In order to do so, first load the first ASF file and AMC file, then load the second ASF file (may be same as first, but must be still loaded), and the second AMC file. You will see the two motion on top of each other.

Adjusting playback speed

By default, the player plays the motion at normal speed, that is, 120 frames per second. In order to slow down the motion, enter the corresponding value into the "Speed" box. For example, to make the playback 5x slower, enter 0.2 into the "Speed" box.

Displaying the frame of a joint

You can display the frame of a joint by entering its index (positive integer) into the "Show Joint" textbox. The default value is "-1" (no joint shown).

Interpolation

Use the provided "interpolate" driver application to perform the interpolation. This driver loads a skeleton and motion file, performs interpolation, and outputs the resulting AMC motion file. You can then load this AMC file into the mocap player for visualization. You can use the "Reload" button to reload the AMC motion (useful, for example, when tweaking your interpolator class). See the command line options for how to execute various interpolation techniques. For example, to perform Bezier Euler interpolation, using N=10, use:

./interpolate 131-dance.asf 131_04-dance.amc b e 10 131_04-dance-be-N10.amc


Tips


Extra credit

Here are some extra credit ideas:

Also, you might be awarded credit for any other creative or interesting solution to an issue in your implementation, as long as it is well-documented in your readme file. So, if you can think of any interesting feature, feel free to implement it.

Please note that the amount of extra credit awarded will not exceed 20% of this assignment's total value.


by Jernej Barbic, Yili Zhao and Yijing Li, USC; credits also to the CMU graphics lab