關(guān)于數(shù)學(xué)公式語音項目的調(diào)研.ppt

關(guān)于數(shù)學(xué)公式語音 項目的調(diào)研,報告人：彭云輝 2010-4-8,綱要,國外關(guān)于數(shù)學(xué)語音的一些相關(guān)的研究 這些項目表示數(shù)學(xué)公式所采用的語言/格式 相關(guān)項目的大體思路及意圖 如何消除歧義 項目的優(yōu)點和局限性及對項目的改進意見,主要相關(guān)項目及對應(yīng)機構(gòu)或人物,另外，還有一些機構(gòu)、團體對數(shù)學(xué)公式語音作了深入的研究，并取得了一定的成果。如University of California at Berkeley(How can we speak math/Math Speak & Write, a Computer Program to Read and Hear Mathematical Input) 和 School of Computing, Dublin City University (Mathematics: How and What to Speak)等。,RETURN,數(shù)學(xué)表達式源語言,RETURN,an excellent prototype for speaking mathematics LaTeX to audio documents. Can speak both literary texts and highly technical documents that contain complex mathematics. the adequacy of the audio rendering depends on how well the electronic document captures the essential internal structure of the information. produced a structured representation and an audio formatting language (AFL) to provide an interactive environment for listening to and browsing technical documents. uses the Emacs(文本編輯器) front-end (Linux).,有關(guān)AsTeR( Audio System for Technical Readings ) (Raman 1994),AsTeR（續(xù)）,creates an internal representation easier,used to help the audio rendering,Mathtalk的大體思路及意圖 1. a set of rules to insert prosodic cues into spoken algebraic expressions. 2. analyzed the way mathematics teachers speak mathematical expressions and integrated these natural voice inflections（音調(diào)變化）. 3. only insertion of prosodic cues (pitch, amplitude, and pauses) into computer-spoken mathematical expressions ,none insertion of lexical cues.,creates the necessary HTML/XML tags for visually-impaired and blind users to use their current screen-reading tools (e.g. JAWS and Window-Eyes for Windows) to read HTML and MathML/XML pages that contain math expressions, to read them in a spoken language.,This markup is not displayed in the browser. Only the MathML visual markup, or a PNG image, or a LiveMath Plug-In interactive image - whatever the author intended, is shown. The “MathSpeak This“ function makes it possible to hear the expression read during the creation/editing process,大體思路： 1. deriving Braille and Spoken Output from LaTeX Documents 2. render spoken mathematics from MathML using prosodic features such as pauses and speaking rate . 3. the use of prosody in synthesized speech to indicate nesting structure. 主要意圖： To take a subset of LaTeX and produce both Braille and Spoken out from it. To accurately model a document and to present this to the blind user using a simple and intuitive interface. To harness the capabilities of synthetic speech devices to give more meaningful spoken output to the user.,TechRead的大體思路及意圖,AudioMath的大體思路及意圖 made use of its own database of prosodic rules in the generation of the spoken expressions. Available in 4 different ways: - ActiveX DLL - .NET component - CGI interface - Executable EXE,Auto-Discovery (the “brain” of the operation that recognizes or identifies elements in the document and calls the respective conversion modules ) Numerals (conversion of several types of numeric forms) Abbreviations Acronyms Network References Mathematical (MathML expressions ),6 modules for the conversion part,AudioMath設(shè)計流程圖,MathPlayer,a plug-in to Microsoft Internet Explorer (IE) and Adobe Acrobat/Reader that renders MathML visually. is able to dynamically display a mathematical expression according to its font and the color set, users can choose the most suitable font or color scheme for their reading needs. For example, visually impaired readers are likely to set a large font and high color contrast.,上述公式在MathPlayer中的讀法為： cap U bar sup h equals one minus exponent open minus fraction 8 cap T sup h over end fraction close,讀法： equals ln open fraction n over s end close plus open fraction k sup h over k prime sup h end fraction close ln open s close minus zero point seven five plus open two l z minus z squared close fraction k sup h over q sup w end fraction.,MathSpeak Project的大體思路及意圖 The project is one of the proposed methods, consisting of a group of rules to dictate mathematical contents. However it is not a standard and it is intended to serve blind people that want to transcribe their documents into Nemeth Code 18, and later on into Braille.,RETURN,如何消除歧義,兩大策略：,Use of lexical indicators (a) x plus begin fraction one over x end fraction minus one (b) begin fraction xplusone over x end fraction minus one Use of prosodic indicators(pauses, modifications of pitch and tempo, rhythm and tone) (a) x plus one over x minus one (b) “ xplus one over x minus one,在消除歧義這一部分，MathPlayer沒什么優(yōu)勢，而AudioMath在這一方面做得很不錯，其余的一些相關(guān)軟件也沒什么出眾的地方，下面著重談?wù)凙udioMath,AudioMath(葡萄牙語),Lexical Square root of power base a exponent two, end of power, plus power base b exponent two, end of power, end of radicand Prosodic Square root of (LP) a squared (SP) plus b squared (LP) end of radicand,AudioMath tone rules:,1- Rising tone: used when a lower hierarchical level is starting. (root of) 2- Falling and Rising tone: used to mark the smaller separating pause. (a squared) 3- Falling tone: used when level is ended. (b squared) 4- Emphatic Falling tone: used at the end of the expression that simultaneously is the higher hierarchical level (end of radicand).,LP,SP,LP,RETURN,AudioMath優(yōu)點：,supports usermode options. An example ： 1.25 one point twenty five OR one point two five,Future Work of AudioMath - Complete the support for MathML Content Markup - Study in more detail mathematical prosody - Implement a proper blind tool - Add more languages - Enhancements on XHTML support - Implement SAPI, SSML support for TTS technologies,MathPlayer局限性,the use of tables and the representation of matrices and the possibility of some ambiguous readings no math formulae navigation support. gets complicated with complex math expressions no provision for any kind of user adapted preferences scheme（usermode） has ambiguous rendering in some mathematical expressions. Does not use prosody to render mathematical expressions by speech output. It generates text strings made up of the names of mathematical symbols and commas and periods to set pauses.,MathPlayer優(yōu)勢,allows web browsers users to copy a MathML expression and paste it in a MathML-aware program. This is particularly useful for computation, but might also be useful when used in conjunction with other software aimed at making math accessible (e.g. the LAMBDA system) or with mainstream applications used to process scientific documents (e.g. MathType or Scientific Notebook).,Changes in MathPlayer 2.2,MathPlayer 2.2 (released February 2010) is an upgrade and includes the following: Significantly improved font handling and rendering: Improved support for STIX, Cambria, and other Unicode fonts. Improvements for anti-aliased rendering. Better protection against fonts that contain errors in their tables. More characters are displayed. Improved performance when Internet Explorers zoom is not 100%. Improved compatibility with ASCIIMathML. Fixed bugs with content MathML (handling of , “Copy MathML“),Future work of MathPlayer,MathPlayers speech rules are based upon a pattern matcher/rule system. The rules are able to specify synchronization points and prosody in addition to text to speak. The rules provide a great deal of flexibility and allow users to match structures such as limits and integrals so that they are spoken in the customary manner rather than treating them as general expressions with limits and/or scripts.,Future work of MathPlayer （續(xù)）,The downside to this power is that