[Ltg] HAIL/SALS-SIG Seminar (24th April): Dr Mark Dras, Macquarie University

[Andrew Lampert]

                      H.A.I.L. Seminar series
                          CSIRO ICT Centre
                   http://www.ict.csiro.au/HAIL/

                                AND

                              SALS-SIG
        Sydney Area Language and Speech Special Interest Group
            http://www.clt.mq.edu.au/Events/SALS-SIG.html


Title:          GLEU: Automatic Evaluation of Sentence-Level Fluency

Speaker:        Dr Mark Dras
                Senior Lecturer
                Centre for Language Technology
                Macquarie University

Date:           Tuesday 24th April 2007 at 11am

Location:       CSIRO ICT Centre,
                Building E6B, Macquarie University.

                See <http://www.ict.csiro.au/HAIL/location.htm> for details.


Video:          We are trialling live VIDEO streaming of seminars.

                At the nominated time above, point your browser at:
                <http://webcast.nsw.csiro.au/httpfs/ICT/HailSeminar/live.asx>


Abstract

In evaluating the output of language technology applications -- MT, natural language generation, summarisation -- automatic evaluation techniques generally conflate measurement of faithfulness to source content with fluency of the resulting text. We have developed an automatic evaluation metric to estimate fluency alone, by examining the use of parser outputs as metrics, and we show that they correlate with human judgements of generated text fluency. From this we have developed a machine learner based on these, which performs better than the individual parser output metrics, approaching a lower bound on human performance. We have also investigated the effect on the metric of different language models for generating sentences, and show that while individual parser metrics can be 'fooled' depending on generation method, the machine learner provides a consistent estimator of fluency.

This is joint work with Andy Mutton and Stephen Wan.


Short resume

Mark Dras is a senior lecturer in the Department of Computing at Macquarie University. He's interested in machine translation, paraphrase, and formal grammars, notably (courtesy of a postdoc at the University of Pennsylvania) Tree Adjoining Grammar.


----------------------------------------
The HAIL Seminars' URL:
http://www.ict.csiro.au/HAIL/

Contacts:       Andrew Lampert
Address:        CSIRO HAIL Seminars,
                c/o Andrew Lampert,
                Locked Bag 17,
                North Ryde NSW 1670

Phone:  (02) 9325 3100
Email:  Andrew.Lampert at csiro.au


*Administration*
-----------------------------------------------------------------
* To leave the list, send the message "unsubscribe HailSeminars" to
the list server address <majordomo at nsw.cmis.csiro.au>.
The subject line of your e-mail message will be ignored.
==================================================================