Description

In addition to our role in developing transfer learning algorithms for the DARPA Transfer Learning program, our group will also be providing support in the evaluation of the algorithms produced by the technology development teams within this program. These evaluations will help the teams to better evaluate and hopefully improve their algorithms by providing them with useful data analysis software tools and external performance validation.

The slides below describe the data format required by our data analysis program. The final link provides access to the actual code that runs this data analysis. The code is written in LISP and documentation is included in the file.

Important: Please read the summary and assumptions regarding input data, given at the top of the analysis code.

• Data Format (PPT slides)
• Explanation of statistical hypothesis testing method (Slides in PDF format)
• UCB's Y1 metric proposals (Slides in PDF format)

• Y2 Evaluation Code (LISP file , Sample data file, Sample data file). Last updated Sept. 27, 2007.
  • Ensure your data files include at least 7 replications of each condition (unlike the sample files).
  • "k" is an input parameter that can be passed to the evaluation function. According to Dan's slides, this "k" should be determined jointly with the evaluation team. If "k" is not specified, all data is used.

• Y2 Evaluation Code notes and instructions (coming soon). See https://oblinger.sainc.com/confluence/display/TL/TL.RegretIssue for an overview.

Personnel

• Prof. Paul R. Cohen
• Yu-Han Chang
• Clayton T. Morrison

Funding

This work was supported by the Defense Advanced Research Projects Agency (DARPA) contract through the Transfer Learning program via contract N00173-05-1-G035.